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INTRODUCTION 


This  Technical  Report  is  part  of  the  research  activity  conducted  to  support  the  US  Air 
Force's  Armstrong  Laboratory  Situation  Awareness  Integration  (SAINT)  team's  efforts.  The 
SAINT  team  was  formed  to  organize  the  research  activities  of  separate  groups  within  the 
Armstrong  Laboratory  to  investigate  Situation  Awareness  (SA)  and  its  relevance  to  the  Air  Force. 

The  Technical  Report  is  divided  into  two  main  sections;  Section  I  presents  four  papers 
reviewing  aspects  of  the  S  A  concept  and  research.  Section  II  is  an  annotated  bibliography  of 
published  S  A  papers. 


Section  I  -  Papers 

Section  I  begins  with  a  paper  by  Capt  Cynthia  Dontinguez  that  addresses  the  controversial 
issue  of  a  definition  for  SA.  The  need  for  an  agreed-upon  definition  has  been  lamented  in  a  very 
large  proportion  of  the  SA  research  papers.  Many  researchers  have  already  attacked  this 
problem,  Capt  Dominguez's  paper  will  review  these  earlier  attempts  to  determine  whether  a 
consensus  definition  can  be  constructed. 

The  second  paper,  by  Dr.  Michael  Vidulich,  was  originally  written  during  the  early  stages 
of  the  SAINT  team's  response  to  S  A  concerns  raised  by  the  Air  Force's  Chief  of  Staff  The  paper 
attempted  to  describe  some  of  the  relevant  issues  from  cognitive  psychology  that  could  impact 
upon  an  SA  research  program. 

The  third  paper  was  written  by  Lt  Col  Eric  Vogel.  Lt  Col  Vogel  has  a  strong  background 
in  Air  Force  tactical  airlift  operations.  Inasmuch  as  the  SA  concept  originated  with  the 
operational  community,  it  is  appropriate  that  their  concerns  be  directly  represented  in  this  report. 
Lt  Col  Vogel's  paper  should  be  of  especial  interest  because  it  addresses  SA  in  the  domain  of 
tactical  airlift.  Traditionally,  discussions  of  SA  have  been  dominated  by  its  role  in  a  fighter 
aircraft  cockpit.  In  demonstrating  the  relevance  to  a  new  domain,  Lt  Col  Vogel's  paper  shows  the 
generality  of  the  SA  concept. 

The  final  paper  of  Section  I  is  by  Dr.  Grant  McMillan.  It  provides  a  summary  of  the 
SAINT  team's  initial  research  project  that  responded  to  SA  concerns  raised  by  the  Air  Force  Chief 
of  Staff,  General  McPeak.  This  paper  started  as  an  edited  version  of  a  November  1993 
presentation  to  the  Air  Force  Scientific  Advisory  Board,  but  it  has  been  updated  to  include  the 
data  and  results  that  were  presented  in  a  briefing  to  General  McPeak  in  April  1994.  As  can  be 
inferred  from  the  paper,  this  program  had  several  components  that  are  very  likely  to  be  presented 
as  separate  reports  of  considerably  more  detail.  Nevertheless,  we  feel  that  an  overview  of  this 
large  program  will  be  of  interest  to  the  S  A  research  community. 
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Section  n  >  Annotated  Bibliography 

The  second  section  of  the  report  is  an  annotated  bibliography  of  S  A  papers.  It  must  be 
acknowledged  that  this  bibliography  is  not  exhaustive.  Although  SA  is  a  relatively  recent  term,  it 
would  be  impossible  within  any  reasonably-sized  effort  to  find  and  review  every  paper  that  has 
invoked  the  concept.  Nevertheless,  we  have  attempted  to  collect  papers  from  a  wide  enough 
group  of  sources  to  be  representative  of  the  extent  of  the  SA  concept  within  the  literature  and  to 
review  enough  papers  to  form  a  critical  mass  of  information  for  a  researcher  trying  to  become 
familiar  with  the  field. 

The  accumulation  of  papers  started  with  the  personal  collections  of  the  reviewers.  This 
was  supplemented  by  a  computerized  literature  search  of  the  NASA-RECON,  Defense  Technical 
Information  Center,  PsychINFO,  and  Energy  Science  &  Technology  databases.  The 
computerized  literature  search  was  conducted  by  Michael  Gravelle  and  J.N.  Hecht  of  the  Crew 
Systems  Ergonomics  Information  Analysis  Center  (CSERIAC).  Finally,  manual  surveys  of 
numerous  journals,  proceedings  series,  and  magazines  were  conducted.  In  particular,  several 
years  oi Aviation  Week  &  Space  Technology  were  surveyed  to  collect  examples  of  how  SA  was 
used  by  the  operational,  rather  than  the  scientific,  community. 

Of  the  final  collection  of  papers,  over  200  were  reviewed  and  categorized.  The  citations, 
reviews,  and  lists  of  relevant  keywords  are  presented  in  the  annotated  bibliography.  In  order  to 
assist  in  accessing  the  most  relevant  citations,  the  bibliography  is  followed  by  a  keyword  glossary 
and  two  indexes.  The  keyword  glossary  provides  a  description  of  the  keywords  used  to  classify 
each  of  the  entries  in  the  bibliography.  The  keyword  index  lists  all  of  the  entries,  identified  by 
number,  that  had  a  given  keyword  assigned  to  them.  The  second  index  provides  a  listing  of  all  of 
the  authors  cited  in  the  bibliography  and  the  bibliographic  entries  with  which  each  author  was 
associated. 

The  authors  of  this  report  hope  that  the  community  of  researchers  investigating  SA  will 
find  the  papers  and  bibliography  included  in  the  report  to  be  helpful. 
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SECTION  I  - 
PAPERS 
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CAN  SA  BE  DEFINED? 


Capt  Cynthia  Dominguez 
Crew  Systems  Directorate 
Crew  Technology  Division 

Intrudiactton 

This  paper  undertakes  the  task  of  analyzing  situational  awareness  (SA)  definitions  in  the 
open  literature  in  order  to  establish  the  elements  which  should  comprise  a  definition  of  SA.  One 
consolidating  definition  is  then  presented. 

When  a  new  construct  emerges  and  gains  momentum  in  the  academic  and  applied 
communities  with  such  force  as  has  situational  awareness,  it  is  only  natural  that  those  of  us  who 
pursue  its  meaning  and  its  measurement  should  seek  to  define  it  as  well.  Unfortunately,  the  lack 
of  an  agreed-upon  definition  of  SA  has  itself  been  a  defining  characteristic  of  SA  from  the  start. 
As  was  the  case  with  mental  workload,  there  are  many  definitions,  and  although  the  concept  is 
accepted  as  important  without  qualificadon,  nobody  is  willing  to  accept  anybody  else's  definidon 
(Wickens,  1992). 

This  definition  analysis  effort  is  undertaken  with  a  high  level  of  SA,  at  least  in  relation  to 
current  publications  on  the  subject  of  SA,  since  reviewing  numerous  papers  en  route  to  creating 
an  annotated  bibliography  provides  an  overall  big  picture  of  the  field.  Armed  with  the  thoughts 
and  perspectives  of  a  multitude  of  practitioners  and  researchers,  the  goal  of  this  paper  is  to 
integrate  the  available  information,  to  hash  through  and  distill  current  thinking  in  order  to  cull  out 
the  most  vital  components  of  each  perspective.  Throughout  this  process,  the  aim  is  to  claiify  the 
essence  of  SA  into  a  definition  which  succinctly  encapsulates  its  nature. 

Why  do  we  need  a  definition  of  S  A?  Lack  of  a  standard  definidon  does  not  hinder  those 
in  search  of  a  research  project  involving  SA,  and  certainly  doesn't  hold  back  those  of  us  who  are 
already  involved  in  some  aspect  of  SA  research.  Sarter  and  Woods  (1991)  point  out  that  a 
significant  hindrance  in  any  effort  to  cull  the  critical  components  of  SA  into  a  definidon  is  the 
context-dependent  nature  of  the  concept:  what  is  relevant  information  in  one  flight  situadon  may 
have  little  impact  in  another.  They  imply  that  any  definition  general  enough  to  account  for  the 
context-sensitivity  of  available  flight  informadon  will  probably  be  too  broad  to  be  of  use.  A  broad 
definition  describing  how  cognitive  processes  work  to  achieve  SA  is  certainly  in  danger  of  being  a 
general  description  of  human  information  processing. 

Yet,  in  spite  of  this  pitfall,  if  seems  that  a  condensed  verbal  summary  of  the  general 
process  of  sifting  through  and  using  environmental  and  internal  informadon  In  flight  can  be  a 
critical  tool.  We  must  be  able  to  represent  the  S  A  construct  in  a  manner  that  users,  engineers,  and 
lay  people  can  understand  and  appreciate.  In  addition,  we  need  greater  understanding  of  the 
concept  for  the  purposes  of  moving  forward  with  meaningful  research.  Consider  the  number  of 
SA  definitions  presented  in  Table  1.  Each  definition  may  generate  a  unique  train  of  study  designs 
and  influence  the  choice  of  measures  used  in  these  studies,  as  was  the  case  with  mental  workload. 
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Table  1. 


Definitions  of  SA 


Definitions 


Conscious  awareness  of  actions  within  two 
mutually  embedded  four-dimensional  envelopes. 

(Beringer  and  Hancock,  1989,  p.646) 

A  pilot's  continuous  perception  of  self  and  aircraft 
in  relation  to  the  dynamic  environment  of  flight, 
threats,  and  mission  and  the  ability  to  forecast,  then 
execute  tasks  based  on  that  perception. 

(Carroll,  1992) 

The  ability  to  extract,  integrate,  assess,  and  act 
upon  task-relevant  information  is  a  skilled  behavior 
known  as  'situational  awareness.' 

(Companion,  Corso,  Kass,  &  Herschler, 
1990) 

I'he  accurate  perception  of  the  factors  and 
conditions  that  affect  an  aircraft  and  its  flight  crew. 

(Edens,  1991,  p.  7.  Schwartz,  1993, 
uses  this  definition  with  "during  a 
defined  period  of  time"  at  the  end.) 

The  perception  of  the  elements  in  the  environment 
within  a  volume  of  time  and  space,  the 
comprehension  of  their  meaning,  and  the  projection 
of  their  status  in  the  near  future. 

(Endsley,  1990,  p.1-3) 

The  knowledge  that  results  when  attention  is 
allocated  to  a  zone  of  interest  at  a  level  of 
abstraction. 

(Fracker,  1988,  p.  102) 

The  pilot's  overall  appreciation  of  his  current 
'world.' 

(Gibson  &  Garrett,  1990,  p.7-1) 

One's  ability  to  remain  aware  of  everything  that  is 
happening  at  the  same  time  and  to  integrate  that 
sense  of  awareness  into  what  one  is  doing  at  the 
moment. 

(Haines  and  Plateau,  1992,  p.  43) 

Where  refers  to  spatial  awareness. . .  >vhat 
characterizes  identity  awareness,  or  the  pilot's 
knowledge  of  the  presence  of  threats  and  their 
objectives,  [as  well  as]  engine  status  and  flight 
perf^.mance  parameters.  Who  is  associated  with 
responsibility,  or  automation  awareness;  that  is 
knowledge  of 'who's  in  charge.'  Finally,  when 
signifies  temporal  awareness  and  addresses 
knowledjte  of  events  as  the  mission  evolves. 

(Harwood,  Barnett,  and  Wickens,  1988, 
p.  316) 

The  ability  to  envision  the  current  and  near-term 
disposition  of  both  friendly  and  enemy  forces." 

(Masters,  McTaggart,  and  Green,  1986, 
p.5;  Stiffler,  1987) 

Awareness  of  conditions  and  threats  in  the 
immediate  surroundings. 

(Morishige  and  Retelle,  1985,  p.  92) 
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1  The  ability  to  maintain  an  accurate  perception  of  the 
suiToimding  environment,  both  internal  and  external 
to  the  aircraft  as  well  as  to  identify  problems  and/or 
potential  problems,  recognize  a  need  for  action, 
note  deviations  in  the  mission,  and  maintain 
awareness  of  tasks  performed. 

(Prince  and  Salas,  1993;  cited  in 

Shrestha  et  al.,  1993,  p.lO) 

[Situational  awareness]  means  that  the  pilot  has  an 
integrated  understanding  of  factors  that  will 
contribute  to  the  safe  flying  of  the  aircraft  under 
normal  or  non-normal  conditions. 

(Regal,  Rogers,  &  Boucek,  1988,  p.65) 

Situation  awareness  refers  to  the  ability  to  rapidly 
bring  to  consciousness  those  characteristics  that 
evolve  during  flight. 

(Wickens,  1992,  p.2) 

The  pilot’s  knowledge  about  his  surroundings  in 
light  of  his  mission's  goals. 

(Whitaker  &  Klein,  1988,  p.321) 

The  problem  with  this  divergence  of  effort  is  that  we  are  left  with  no  unifying  construct  on  which 
to  focus  our  efforts.  Thus,  when  challenged  to  assess  SA  in  an  operational  environment,  it  is 
impossible  to  determine  whose  definition,  whose  line  of  research  should  be  used  as  a  basis  for  this 
work.  And,  as  a  side  benefit,  if  by  some  chance  there  becomes  an  "agreed-upon"  definition  of 
SA,  that  fact  in  itself  will  be  important  to  be  able  to  assert  that  the  study  of  SA  is  thus  different 
from  the  study  of  workload. 

Approach.  To  accomplish  this  breakdown  of  the  components  involved  in  SA,  the 
definitions  available  in  the  literature  (listed  in  Table  1)  were  dissected  and  compared.  The 
process  and/or  product  described  in  each  definition  was  identified,  and  each  major  element  of 
each  definition  was  culled  out  and  evaluated.  The  process  of  SA  refers  to  how  SA  is  developed 
and  maintained  during  flight,  while  the  product  is  the  resultant,  elusive  thing  we  call  SA  itself 
(Tenney,  Adams,  Pew,  Huggins,  and  Rogers,  1992).  Product  and  process  of  SA  are  discussed  in 
detail  below.  Common  themes  were  extracted  from  the  definitions;  the  criterion  for  discussing  a 
common  component  or  theme  in  this  paper  was  whether  it  appeared  in  three  or  more  definitions. 

AhJUties 

"Ability"  is  clearly  a  commonly  used  word  in  defining  SA.  Five  of  the  definitions  in 
Table  1  mention  ability.  The  use  of  the  word  "ability"  brings  out  an  important  dichotomy  about 
the  concept  of  SA.  SA  is  used  interchangeably  as  an  ability,  akin  to  "the  right  stuff,"  which  a  pilot 
either  has  or  hasn't  got,  and  as  a  state  of  mind  which  may  be  achieved  or  not  on  a  flight,  or  which 
may  be  lost  at  certain  times  during  a  flight.  Researchers  examine  SA  from  both  perspectives.  The 
Air  Force  and  other  military  services  are  interested  in  a  basic  ability  to  achieve  SA  because  of  its 
potential  impacts  on  both  training  time  and  the  selection  process.  (In  another  part  of  this  report, 
Grant  McMillan  discusses  an  Air  Force  effort  which  looks  at  SA  both  through  ability  measures 
and  moment-to-moment  performance  measures.) 


Even  so,  possessing  the  ability  to  achieve  SA  can  be  undermined  by  several  factors  which 
impact  in-flight  SA.  Examples  of  these  factors  include  fatigue,  lack  of  recent  training,  emotional 
state,  and  mind-altering  substances.  A  pilot  may  have  the  ability  to  perform  the  cognitive 
processes  which  result  in  SA  without  actually  achieving  it  on  a  particular  flight.  Because  quality 
of  SA  is  subject  to  change  from  flight  to  flight  and  even  from  moment  to  moment  during  flight,  a 
definition  which  describes  how  SA  is  attained  in  flight  as  opposed  to  whether  the  pilot  is  able  to 
achieve  it  in  general  is  preferred.  Therefore,  "ability"  will  not  be  included  in  this  definition  of  SA. 

Pfirtgptian/ExtrflQ.tifi]n 

Companion,  Corso,  Kass,  &  Herschler  (1990)  state  that  "the  ability  to  extract,  integrate, 
assess,  and  act  upon  task-relevant  information  is  a  skilled  behavior  known  as  'situational 
awareness.'"  A  definition  of  SA  should  minimally  include  the  "extraction"  component  they 
identified.  Extraction  is  more  tied  into  actively  perceiving  than  using  the  term  perception 
(perception  is  used  by  Edens,  1991;  Endsley,  1990;  Carroll,  1992;  and  Schwartz,  1993).  Both 
terms  lead  to  an  understanding  of  the  situation  through  sampling  the  environment;  this  is  usually 
accomplished  by  scanning  the  airspace,  cross-checking  the  instruments,  and  watching  and 
listening  to  other  crewmembers.  This  element  is  what  Endsley  (1990)  refers  to  as  Level  1  SA: 
"The  pilot  perceives  the  elements  (e.g.  an  aircraft,  a  mountain,  a  warning  light)  that  are  present  in 
the  environment,  along  with  their  relevant  characteristics  (e.g.  color,  size,  speed,  location)."  (p.l- 
2)  Schwartz  (1993)  states  that  SA  begins  with  "accurate  perception...,"  while  Carroll  has  adopted 
a  definition  which  calls  for  "continuous  perception." 

An  insightful  perspective  on  this  element  of  SA  is  given  by  Tenney  et  al.  (1992),  who 
argue  that  current  definitions  of  SA  differ  as  to  whether  they  emphasize  the  product  or  the 
process  of  SA.  "The  state  of  awareness  with  respect  to  information  and  knowledge  is  the 
product.  The  process,  in  contrast,  involves  an  active  and  dynamic  series  of  cognitive  activities." 
(Tenney  et  al.,  1992,  pp.  2-3) 

Tenney  et  al.  state  that  both  types  of  definitions  "capture  the  key  features  of  SA:  an 
understanding  of  the  meaning  of  events  and  the  ability  to  anticipate  the  consequences  of  taking  or 
failing  to  take  paiticular  actions."  (p. They  go  on  to  present  a  unique  view  of  SA,  derived  from 
Neisser's  theory  of  the  perceptual  cyc.e,  which  is  rooted  in  Gibson's  theories  on  the  ecological 
approach  to  visual  perception.  Perception  is  seen  as  a  resonating  cycle  of  sampling  from  the 
environment,  mapping  onto  schema,  and  using  the  schema  to  further  direct  the  sampling;  the 
environment  in  turn  impacts  and  modifies  the  schema  (Neisser,  1976,  cited  in  Tenney  et  al.,  1992, 
p.5).  Endsley  (1990)  likewise  emphasizes  the  active  nature  of  perception;  she  states,  "the  pilot 
SA/decision  making  process  can  be  viewed  as  a  dual  process  whereby  active  schema  and  scripts 
are  dictating  which  information  to  focalize  attention  on,  and  simultaneously  the  presence  of 
certain  objects  or  attributes  in  the  environment  will  activate  new  schema  in  long-term  memory." 

(p.  1-3) 

This  cyclical  process  emphasizes  extraction  over  perception  to  describe  the  active  manner 
in  which  a  pilot  acquires  the  information  he  or  she  needs.  For  example,  on  a  low-level  bomb  run, 
an  experienced  pilot  knows  what  information  is  needed  to  complete  the  run  successfully;  this 
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information  is  actively  extracted  from  instruments,  environment,  other  ciewmembers,  and  from 
internal  knowledge  about  the  mission.  The  pilot  is  not  simply  a  sponge;  query  and  discrimination 
aie  ongoing  continuously.  The  process  of  extraction  reveals  intelligence  wliich  updates  the 
product  SA,  which  in  turn  directs  the  pilot's  attention  where  it's  needed  next.  The  beauty  of  this 
approach  is  that  it  intertwines  the  process  of  developing  SA  with  the  resultant  SA,  demonstrating 
the  interdependence  of  the  two.  By  subscribing  to  the  notion  that  perception  is  actively 
accomplished  through  extraction,  a  persuasive  case  can  be  made  for  a  definition  of  SA  which 
accounts  for  both  the  process  and  product  involved. 

The  Product 

Up  to  this  point,  the  importance  of  information  extraction  in  SA  and  the  usefulness  of 
combining  the  product  and  the  process  of  SA  into  one  definition  have  been  established.  The  next 
element  which  surfaces  from  several  definitions  of  SA  concerns  how  information  extracted  from 
the  environment  is  incorporated  into  the  pilot's  overall  picture,  or  "coherent  internal  world  view" 
(Goodson  et  al.,  1990,  p.2).  This  element  is  referred  to  by  various  authors  as  "comprehension," 
"integration,"  "bringing  to  consciousness,"  and  "integrated  understanding,"  which  bring  to  mind 
the  process  of  melding  previously-known  and  freshly  extracted  information  into  a  cohesive 
picture.  Endsiey’s  3-Level  model  of  SA  (1990,  p.  1-2)  identifies  Level  2  SA  as  "a  holistic  picture 
of  the  environment"  which  integrates  the  perceived  environmental  elements  from  Level  1 .  In  line 
with  the  paradigm  used  to  categorize  SA  definitions  in  this  paper,  this  picture  is  essentially  the 
product  of  SA. 

Many  cognitive  theories  have  been  developed  to  describe  the  processes  of  comprehension, 
integration,  and  understanding  involved  in  attaining  SA.  In  fact,  several  authors  have  written 
extensively  on  the  cognitive  underpinnings  of  the  SA  construct  (e.g.  Endsley,  1990;  Fracker, 

1988;  Tenney  et  al.,  1992;  and  Vidulich,  this  report.)  Throughout  the  literature,  SA  (the  product) 
is  referred  to  as  a  mental  model.  Such  a  mental  model  is  formed  based  on  all  relevant  information 
that  the  pilot  has  absorbed  and  integrated  which  impacts  the  current  flight.  It  combines  earlier 
knowledge  and  experience  with  the  current  flight's  briefing  information  and  up-to-the  minute 
cockpit  status.  The  pilot's  mental  model  is  continuously  being  updated,  and  it  is  used  to  predict 
future  trends.  It  is  a  Gestalt,  according  to  Gibson  and  Garrett  (1990),  which  is  greater  than  the 
sum  of  its  parts.  Sarter  and  Woods  (1991)  state  that  adequate  mental  models  are  a  prerequisite 
for  achieving  SA.  (See  Harwood,  Barnett,  and  Wickens,  1988,  for  an  in-depth  discussion  on  SA 
and  mental  models) 

Although  there  is  controversy  over  the  definition  of  mental  models  (Rowe,  Cooke, 

Neville,  and  Schacherer,  1992),  the  concept  is  useful  in  that  it  facilitates  understanding  of  the 
nature  of  human  information  processing  as  it  applies  to  complex,  dynamic  tasks  such  as  piloting 
an  aircraft.  These  models  are  like  schema,  referred  to  by  Tenney  et  al.  (1992)  above,  in  that  they 
are  developed  and  improved-upon  through  experience  and  training.  Schema  are  conceptual  long¬ 
term  memory  structures  used  to  organize  task-related  and  other  information.  Cues  activate 
schema,  causing  them  to  enter  working  memory  where  they  update  the  working  mental  model. 
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The  lack  of  schema  or  mental  models  greatly  increases  the  mental  workload  for  novice 
pilots.  For  example,  first-time  pilots  often  cite  the  difficulty  of  interpreting  and  making  radio 
calls.  As  a  pilot's  hours  accumulate,  he  or  she  develops  a  conceptual  framework  in  long-term 
memory  for  understanding  and  responding  to  radio  calls;  these  calls,  over  time,  become  almost 
second  nature.  Over  time,  the  processes  involved  become  "automatic,"  requiring  little  effort  and 
posing  no  load  on  working  memory  (Schneider  and  Shiffrin,  1977).  Hartman  and  Secrist  (1991) 
describe  this  process  nicely;  "Automatic  processes  develop  as  a  result  of  long-term,  consistent 
mapping  between  stimulus  arrays  and  response  repertoires."  The  process  of  training  a  pilot  really 
revolves  around  developing  in  the  pilot  a  set  of  knowledge  structures  which  can  be  activated 
quickly,  without  conscious  thought,  and  with  little  or  no  attention,  so  that  the  aircraft  can  be 
operated  in  a  dynamic,  challenging  environment  without  over  stressing  the  pilot's  working 
memory. 

All  of  these  constructs  (mental  mode’s,  schema,  automatic  processing)  have  been  used  to 
describe  how  pilots  achieve  SA,  as  have  other  constructs  for  understanding  information 
processing  which  are  not  mentioned  here.  Unfortunately,  these  constructs  don't  always  converge; 
they  aie  similar  to  the  SA  construct  (and  to  mental  workload)  in  that  there  is  little  agreement 
among  authors  in  how  these  theories  streamline  together,  or  in  how  they  relate  to  each  other.  To 
choose  one  term  such  as  "mental  model,"  "schema,"  or  "automaticity"  to  describe  the 
"integration"  element  for  a  definition  of  SA  would  be  extremely  difficult;  in  addition,  it  would  be 
poorly  understood  by  those  who  are  not  familiar  with  information  processing  literature. 
Developing  a  definition  of  SA  which  is  understood  and  accepted  by  the  pilot  and  aerospace 
communities  is  important.  Presenting  this  element  in  terms  indicating  the  integration  of 
information  into  an  overall  mental  picture  meets  this  criterion,  and  avoids  selecting  from  the 
many  specific  cognitive  constructs  discus.sed  above.  Thus,  this  terminology  will  be  employed  here 
in  a  consolidated  SA  definition. 


Along  with  the  perceptual  cycle  and  internal  world  model,  there  is  one  final  concept  not 
yet  mentioned  which  is  integral  to  any  discussion  of  SA:  the  temporal  dimension.  Temporal 
awareness  has  been  described  as  the  most  critical  element  in  maintaining  SA  (e.g.,  Harwood, 
Barnett,  and  Wickens,  1988;  Sarter  and  Woods,  1991).  "Staying  ahead  of  the  airplane"  is  a 
phrase  often  used  to  summarize  a  pilot's  ability  to  anticipate  and  prepare  for  future  events  (e.g. 
Regal,  Rogers,  and  Boucek,  1988;  Schwartz,  1993). 

The  timeline  which  characterizes  a  flight  or  a  mission  is  relentless;  after  takeoff,  the 
aircraft  will  move  through  space  and  time  whether  the  crew  are  in  pace  with  it  or  not.  Like  the 
operators  of  any  complex  and  dynamic  system,  pilots  constantly  must  think  ahead  to  the  next  step, 
anticipating  future  actions  and  trends.  One  reason  for  this  is  that  unforeseen  occurrences  and 
emergencies  are  inevitable,  and  if  the  crew  are  not  thinking  ahead  of  the  timeline,  they  can  easily 
become  mired  in  a  catch-up  game  which  can  create  panic  and  a  snowball  of  mistakes.  In  a  non- 
attribution  system  for  reporting  unsafe  conditions  in-flight,  pilots  themselves  report  that  a  lack  of 
temporal  awareness  frequently  leads  to  unsafe  situations,  and  that  this  aspect  of  piloting  merits 
more  study  (Sarter  and  Woods,  1991). 
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As  to  how  this  temporal  dimension  should  be  included  in  an  all-encompassing  definition  of 
SA,  consider  the  definitions  of  Endsley  (1990)  and  Carroll  (1992).  Endsley,  as  might  be  expected 
following  the  description  of  SA  Levels  1  (perception)  and  2  (a  holistic  picture  of  the  environment) 
above,  completes  the  breakdown  with  Level  3  SA,  the  ability  to  translate  a  pilot's  mental  model 
into  projections  of  the  future.  Indeed,  this  concept  forms  the  last  phrase  of  her  fonnal  definition 
of  SA  (See  Table  1).  The  definition  from  Carroll  (1992)  similarly  concludes  with  "the  ability  to 
forecast,  then  execute  tasks  based  on  that  perception."  (p.  6)  These  perspectives  are  related  to 
anticipating,  thinking  ahead  of  the  aircraft,  and  predicting  the  implications  of  currently-known 
information.  (The  past,  or  gone-by  segment  of  the  timeline  is  also  important  in  SA,  insofar  as  it 
contributes  to  the  pilot's  current  picture.  Because  it  is  inherent  in  the  description  of  what  makes 
up  the  mental  model,  it  is  not  proposed  for  inclusion  in  a  definition.)  Because  thinking  ahead, 
forecasting,  and  projecting  into  the  future  are  so  important  in  the  development  and  maintenance  of 
superior  SA,  words  to  this  effect  should  be  included  in  a  definition  of  SA. 

Putting  it  together... 

Up  to  this  point,  we  have  established  four  pieces  of  information  which  should  be  included 
in  a  comprehensive  definition  of  SA.  They  are  (1)  extracting  information  from  the  environment; 

(2)  integrating  this  information  with  relevant  internal  knowledge  to  create  a  mental  picture  of  the 
current  situation;  and  (3)  using  this  picture  to  direct  further  exploration  in  a  continual  perceptual 
cycle,  as  well  as  to  (4)  anticipate  future  events. 

Keeping  these  four  points  in  mind  while  reviewing  all  of  the  existing  definitions,  the 
following  definition  is  proposed: 

Continuous  extraction  of  environmental  information,  integration  of  this 

information  with  previous  knowledge  to  form  a  coherent  mental  picture,  and  the 

use  of  that  picture  in  directing  further  perception  and  anticipating  future  events. 

The  process  of  SA  includes  extraction,  integration,  and  the  use  of  the  mental  picture, 
while  the  picture  itself  is  the  product  SA.  Thus,  the  product  SA  can  be  poor  or  good,  depending 
upon  whether  the  process  is  successful  in  putting  together  a  picture  which  is  accurate  and 
complete. 

Of  all  the  definitions  listed  in  Table  1,  the  one  above  is  most  similar  to  Endsley's  (1990) 
and  Carroll's  (1992).  Endsley's  work  de.scribes  perception  as  an  active  and  cyclical  process,  but 
that  distinction  is  not  brought  out  in  her  definition.  Carroll's  definition  is  widely  used  in  Air  Force 
circles,  but  gives  no  mention  to  the  integration  of  perceptual  activity,  which  is  a  key  factor.  As  a 
result,  neither  of  these  definitions  was  adopted  here. 

Conclusions 

This  paper  is  an  attempt  to  describe  the  thought  process  involved  in  developing  a 
definition  of  SA  which  reflects  current  thinking  and  which  is  void  of  psychological  or  military 
jargon.  It  is  focused  on  pilot  SA,  but  could  easily  be  applied  to  operators  of  other  complex 
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systems,  such  as  air  traffic  controllers  or  nuclear  power  plant  operators.  It  would  be  naive  to 
expect  SA  researchers  worldwide  to  read  this,  scratch  their  heads,  and  say  "Hmm.  Yes,  this  is  it!" 
Nevertheless,  for  the  reasons  outlined  in  the  introduction  to  this  paper,  it  is  important  that  an 
effort  be  made  to  combine  the  ideas  of  the  many  people  working  to  define,  study,  and  measure 
SA.  We  are  a  group  in  search  of  understanding,  in  search  of  SA,  as  to  what  to  do  about  this 
concept  which  has  taken  the  aviation  community  by  storm. 

Kantowitz  and  Casper  (1988,  p.  162),  in  a  chapter  on  workload  in  aviation,  state:  "There 
is  no  empirical  technique  for  proving  a  definition.  Eiefinitions  are  not  correct  or  incorrect,  but 
rather  more  or  less  useful.  Ultimately,  individuals  must  decide  which  definition  of  workload  will 
be  most  helpful  in  accomplishing  their  own  goals."  I  think  these  words  can  be  well  applied  to  SA 
as  well.  I  apologize  for  the  implication  inherent  in  this  paper  that  other  definitions  are  not  correct 
or  useful.  Many  of  these  definitions,  Endsley's  in  particular,  have  been  used  repeatedly  by  other 
researchers  as  highly  useful  standards.  All  definitions  have  been  useful  to  those  who  have  written 
them  as  a  minimum  in  providing  direction  and  understanding.  As  to  where  we  go  from  here,  there 
are  many  possibilities.  Flach  (1993)  presents  an  exciting  challenge  to  the  human  factors 
community  to  leave  behind  the  approaches  which  were  employed  with  mental  workload.  He 
suggests  using  SA  as  an  opportunity  to  create  a  "theory  of  situations,"  a  positive  framework 
within  which  to  study  how  humans  perform  in  the  context  of  specific  complex  and  dynamic  work 
environments.  Pew  (1994)  also  puts  emphasis  on  understanding  what  is  meant  by  the  term 
"situation."  Researchers  at  the  RAF  Institute  for  Aviation  Medicine  have  long  been  forging  ahead 
with  quality  research,  considering  SA  as  a  global  paradigm  through  which  to  study  the  pilot- 
vehicle  interface  (Taylor  and  Selcon,  1990).  This  work  has  bypassed  the  definitional  debate, 
instead  preferring  to  empirically  determine  the  nature  of  SA  (Taylor,  1990).  Another  interesting 
tact  is  studying  the  linkage  between  expertise  and  SA  (see  Crane,  1992;  Flach,  1993;  Sarter  and 
Woods,  1991 ;  and  Tenney  et  al.,  1992  for  more  on  this  subject). 

Researchers  in  the  field  of  SA  stand  on  the  brink  of  work  which  could  have  great  impact 
on  the  future  design  of  cockpits  and  training  programs,  not  to  mention  man-machine  interface  of 
numerous  other  crew  systems.  A  definition  has  been  presented  which  hopefully  can  move  us 
beyond  one  point  of  contention  in  the  study  of  this  interesting  construct. 
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COGNITIVE  AND  PERFORMANCE  COMPONENTS  OF  SITUATION  AWARENESS; 

SAINT  Team  Task  One  Report 

Michael  Vidulich 
Crew  Systems  Directorate 
Human  Engineering  Division 

The  present  paper  is  intended  to  outline  the  relevant  psychological  issues  for  guiding 
situation  awareness  (SA)  research  in  the  Air  Force.  Specifically,  it  is  an  opening  statement  for 
guiding  the  initial  research  of  the  Armstrong  Laboratory's  Situation  Awareness  Integration 
(SAINT)  team. 

In  order  to  produce  the  document  in  a  timely  fashion  the  literature  review  portion 
emphasized  two  sources  of  information:  (a)  combining  information  from  the  diverse  members  of 
the  SAINT  team  itself;  and  (b)  exploiting  recent  literature  reviews  on  relevant  topics.  The  SAINT 
team  members  represented  a  wide  variety  of  backgrounds  and  research  orientations.  All  members 
have  active  research  projects  on  SA  or  related  topics.  While  not  completely  exhaustive,  the 
information  from  the  SAINT  team  members  was  considered  to  be  a  solid  cross-section  of  the 
current  thinking  regarding  SA.  This  base  was  supplemented  by  using  literature  reviews  of  the 
vital  areas.  Most  notably,  CSERIAC  has  recently  completeo  a  State  of  the  Art  Review  (SOAR) 
on  the  cognitive  issues  associated  with  operator  strategies  in  managing  workload  in  dynamic 
environments.  The  conceptual  framework  developed  in  this  review  has  great  relevance  to  SA 
research.  As  necessary,  the  primary  sources  identified  as  critical  in  (a)  or  (b)  were  directly 
accessed. 

P^finittonaMssww 

A  precise  and  universally  acceptable  definition  of  anything  is  always  difficult  in  science, 
especially  at  the  beginning  of  a  new  research  endeavor.  Nevertheless,  it  is  essential  to  constrain  a 
concept  somewhat  before  any  cohesive  research  program  can  be  outlined.  As  a  starting  point  the 
SAINT  team  has  decided  to  use  the  following  definition  suggested  by  Carroll  (1992): 

Situation  Awareness  (SA)  -  A  pilot's  continuous  perception  of  self  and  aircraft  in 
relation  to  the  dynamic  environment  of  flight,  threats,  and  mission,  and  the  capability  to 
forecast,  then  execute  tasks  based  on  the  perception. 

Two  vital  aspects  of  this  definition  need  to  be  highlighted.  First,  despite  the  name.  SA 
involves  much  more  than  just  simple  awareness  of  the  situation.  Good  SA  implies  a  capability  to 
respond  appropriately,  not  just  the  possession  of  an  accurate  assessment.  It  is  conceivable  that  a 
person  could  be  well  aware  of  eveiy  aspect  of  the  situation  at  the  present  point  of  time  and  still  be 
incapable  of  acting  adaptively.  The  term  SA  implies  not  only  understanding  the  current 
disposition  of  actors  within  a  situation  but  reasonable  projections  of  future  trends.  This 
understanding  is  assumed  to  be  necessary  for  the  selection  or  creation  of  plans  that  will  result  in 
an  operator  achieving  his/her  goals.  In  psychological  terms,  this  means  that  SA  involves  more 
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than  perception  or  pattern  recognition:  it  doubtless  requires  use  of  all  the  higher  cognitive 
functions  a  person  can  bring  to  a  task. 

Second,  despite  the  focus  on  the  pilot  and  an  aircraft,  the  basic  approach  to  SA  can  be 
applied  to  operators  of  almost  any  type  of  system.  This  paper  and  much  of  the  first-year  research 
of  the  SAINT  team  will  focus  on  the  SA  issues  involved  in  a  pilot  engaged  in  visually  guided  air- 
to-air  combat  because  this  is  one  of  the  most  challenging  arenas  for  testing  S  A  tools  of  any  sort 
and  it  is  the  cutting-edge  concern  for  the  Air  Force.  However,  we  cannot  afford  to  overlook  the 
fact  that  SA  is  an  element  in  other  mission-critical  jobs.  The  modem  pilot  is  much  more  likely 
than  before  to  be  part  of  a  real-time  network  of  components  trying  to  achieve  air  superiority.  For 
example,  the  pilot  will  often  be  guided  to  a  favorable  situation  by  an  AW  ACS  weapons  director. 
This  AW  ACS  weapons  director  will  be  using  a  very  crowded  display  to  guide  the  air-battle  and  it 
can  be  assumed  that  the  enemy  aircraft  will  be  trying  their  best  to  confuse  the  AW  ACS  weapons 
director.  So,  SA  is  a  relevant  topic  to  this  operator  as  well.  If  the  AW  ACS  weapons  director's 
SA  is  low,  it  may  be  impossible  for  the  pilot  to  compensate. 

The  Psychology  of  SA  in  Air-to-Air  Combat 

The  air-to-air  combat  arena  is  certainly  among  the  most  challenging  SA  environments  that 
is  engaged  in  by  humans.  The  physical  environment  itself  is  a  very  difficult  one,  especially  if  high- 
G  maneuvers  are  required.  The  relevant  stimuli,  usually  visual,  are  often  very  weak  and  brief. 

The  geometry  relating  all  of  the  active  participants  can  be  very  complex  and  rapidly  changing. 

The  motor  demands  of  operating  all  of  the  myriad  controls  available  to  a  pilot  is  a  task  requiring 
daunting  precision.  Furthermore,  the  task  is  performed  for  the  highest  stakes  possible.  Not  only 
is  the  individual  pilot's  life  possibly  forfeit  if  performance  is  sub-par,  but  the  lives  of  his  or  her 
colleagues  and  the  mission  goals  are  also  at  risk.  All-in-all,  it  seems  likely  that  any  perceptual/ 
cognitive  component  might  be  important  in  such  a  demanding  environment.  But  it  is  not  helpful 
to  say  we  should  study  everything,  some  selection  is  necessary.  As  a  starting  point,  the 
psychological  issues  can  be  divided  into  two  basic  types:  (1)  perceptual-motor  issues,  and  (2) 
cognitive  issues. 

Pgrttptwal-Mptor  Isswts 

In  this  context,  the  tenn  "perceptual-motor"  is  intended  to  connote,  in  a  crude  way,  the 
relatively  peripheral  psychological  processes  used  to  get  information  into  or  out  of  the  human 
operator.  As  a  first  cut,  these  processes  include:  sensory  acuity,  perception  and  partem 
recognition,  and  motor  control.  Each  of  these  will  be  discussed  in  turn. 

Sensory  Acuity.  We  can  assume  as  a  given  that  anyone  selected  as  a  potential  Air  Force 
pilot  will  not  have  notable  deficiencies  in  their  sensory  capabilities.  The  question  to  be 
considered  here  is  whether  unusual  sensory  capabilities  are  to  be  desired  as  a  means  of  improving 
operational  SA. 

The  most  obvious  sense  modality  to  consider  is  vision.  Historically,  the  detection  of 
enemy  aircraft  was  done  by  sight  and  the  first  pilot  to  spot  the  other's  aircraft  had  a  huge 


18 


advantage.  Used  properly,  this  advantage  was  often  translated  into  an  approach  and  attack  before 
the  other  pilot  realized  that  a  threat  existed.  It  is  estimated  that  80%  of  air  combat  kills  in  World 
War  I  and  World  War  II  occurred  without  the  victim  realizing  that  they  were  under  attack  until 
their  opponent  opened  fire  (Spick,  1988). 

Not  surprisingly,  many  pilots  of  this  era  attribute  some  of  their  success  to  good  eyesight. 
For  example,  Chuck  Yeager  points  out  that  his  20/10  vision  gave  him  a  considerable  edge  in 
World  War  II  (Sipe  and  Weksler,  1992).  This  has  even  been  implemented  as  a  feature  in  the  air 
combat  video  game  that  bears  his  name.  A  player  can  activate  "  Yeagervision"  and  a  2x  zoom 
allows  him  or  her  to  see  detail  twice  as  far  away  as  normal.  Another  anecdotal  example  would  be 
Saburo  Sakai's  discussion  of  the  leading  World  War  II  Japanese  ace,  Hiroshi  Nishizawa;  "Even  his 
eyesight  was  unusual.  Where  we  could  only  see  sky,  Nishizawa,  with  almost  supernatural  vision, 
could  catch  the  specks  of  enemy  planes  still  invisible  to  us."  (Sakai,  1958,  p.  84). 

In  the  laboratory,  there  have  been  some  intriguing  correlations  between  visual  capabilities 
and  flight  performance.  For  example,  Kruk,  Regan  and  Beverly  (1983)  found  that  various  tests  of 
super-threshold  motion  were  con  elated  with  several  aspects  of  performance  in  a  flight  simulator. 
Most  relevant  to  the  present  discussion,  Kruk  et  al.  (1983)  detected  a  relationship  between  the 
ability  to  discriminate  different  rates  of  expansion  of  a  test  flow  pattern  and  performance  in  a 
simulated  bombing  task. 

Unfortunately,  there  is  little  evidence  that  simple  visual  acuity  separates  the  ace  from  the 
regular  pilot  in  the  air-to-air  arena.  One  could  also  argue  that  the  current  trends  in  air  combat  are 
likely  to  reduce  the  pilot's  dependence  on  their  own  visual  acuity.  In  the  recent  Gulf  War,  a  large 
proportion  of  the  sorties  occurred  at  night.  Much  of  the  action  was  guided  by  radar  information 
(either  from  the  aircraft's  own  radar  or  from  an  AWACS  aircraft),  and  the  participants  were  often 
separated  by  distances  that  would  have  made  vision  difficult  to  use  even  in  the  daylight.  For 
example,  in  a  discussion  of  an  air-to-air  kill  near  Baghdad  by  Air  Force  pilot  Captain  Steve  Tate, 
the  formation  is  described  with  Tate's  wingman  being  about  35  miles  behind  him,  and  the  actual 
engagement  with  an  Iraqi  Mirage  F-I  started  at  16  miles  with  a  radar  lockup  and  "closed"  to 
about  12  miles  when  he  fired  the  Sparrow  missile  (Ewing,  1992).  According  to  the  story,  the  first 
visual  look  Captain  Tate  got  of  the  Iraqi  aircraft  occurred  when  the  Sparrow  missile  detonated  at 
about  4  miles  away,  illuminating  the  pieces  of  the  exploding  aircraft.  Unusual  visual  acuity 
obviously  did  not  play  a  critical  role  in  this  engagement  nor  are  the  attributes  of  this  mission 
unusual  in  light  of  current  technology  and  tactics. 

Nevertheless,  with  the  current  trend  towards  stealth  aircraft,  it  would  be  presumptuous  to 
assume  that  night-time  radar  engagements  will  always  be  the  norm.  It  is  not  implausible  to 
foresee  missions  that  will  require  a  pilot  to  use  the  basic  eyeball  as  the  principal  search  device. 
Therefore,  visual  acuity  is  probably  a  worthy  consideration  for  evaluating  a  person's  capability  to 
maintain  SA. 

It  is  also  worth  noting  that  there  may  be  a  training  aspect  to  visual  acuity  as  applied  to 
aviation.  Chuck  Yeager  suggests  that  most  people  looking  out  of  the  cockpit  will  have  a 
tendency  to  focus  at  about  18  feet.  It  was  essential  for  a  fighter  pilot  to  learn  to  voluntarily  focus 


19 


the  eyes  to  infinity  while  searching  the  sky  for  enemies  (Iverson,  1991).  Spick  (1988)  cites  ace 
George  Buerling  as  another  example  of  this.  Buerling  would  practice  flicking  his  eyes  around  a 
room  and  focusing  on  objects  at  different  distances  quickly,  "so  that  when  you  look  around  the 
sky  once,  you  pick  up  anything  that’s  out  there"  (p.  1 17). 

Perception  and  Pattern  Recognition.  It  is  often  difficult  in  practice  to  cleanly 
distinguish  between  sensory  processing  and  perceptual  processing  since  even  the  most  peripheral 
sensory  processing  is  influenced  by  meaningful  patterns.  However,  at  this  level  of  the  discussion 
we  will  focus  on  processing  that  is  involved  with  the  picliminary  assignment  of  stimulus  meaning. 

Quickly  "sizing-up"  a  situation  seems  like  a  very  adaptive  trait  for  a  fighter  pilot. 

Rereading  the  above  quote  about  Nishizawa,  the  emphasis  could  be  placed  on  near-threshold 
perceptual  processing  rather  than  visual  acuity.  Stories  of  e.vceptional  pilots  having  an  intuitive 
feel  for  the  geometry  of  an  air  battle  going  on  around  them  are  especially  common  for  World  War 
I  aces  (Spick,  1988). 

Of  a  more  scientific  nature,  Youngling,  Levine,  Mochamuk,  and  Weston  (1977)  noted  that 
one  of  the  selection  tests  used  in  World  War  II  was  the  hidden  figure  detection  test  and  this  test 
had  one  of  the  highest  correlations  with  training  program  survival. 

More  recently,  verbal  protocol  evaluations  have  been  conducted  to  compare  novice  and 
expert  Air  Force  pilots  conducting  simulated  air  intercepts  (Blackman,  Sullivan,  and  Seidler,  in 
preparation;  Sullivan  and  Blackman,  1991).  Among  the  findings  of  this  research  is  that  expert 
pilots  tend  to  attend  more  to  the  overall  geometry  relating  all  players,  while  novices  will  tend  to 
focus  on  a  single  opponent  to  engage.  In  other  w'ords,  the  perception  of  the  situation,  as 
indicated  by  verbal  protocols,  tend  to  imply  that  novices  and  experts  perceive  the  situation 
difterently.  As  structured  in  the  above  experiments,  this  difference  in  perception  would  appear  to 
be  the  result  of  training.  However,  it  is  certainly  plausible  that  people  differ  in  their  abilities  to 
make  the  transition  to  perceiving  the  more  complex  geometry. 

Overall,  the  speed  and  accuracy  of  the  pilot’s  perceptual  processes  (especially  those  involved  in 
detecting  complex  geometrical  relationships)  are  likely  to  be  critical  components  in  the  acquisition 
and  maintenance  of  pilot  SA. 

Motor  Control.  Flying  an  aircraft  well  requires  good  motor  control.  Maintaining  a  good 
flight  path  often  requires  precise  tracking  behavior.  A  pilot  attempting  to  shoot  down  another 
aircraft  with  a  gun  is  involved  in  a  tracking  task  that  requires  exceptional  precision.  This  task 
might  become  rare  in  the  future,  though.  In  the  Gulf  War  there  were  only  two  air-to-air  kills 
using  a  gun.  Both  were  A- 10  aircraft  shooting  down  helicopters  with  their  30  mm  cannon. 
Missiles  are  starting  to  live  up  to  their  potential  and  air-to-air  shooting  is  likely  to  diminish.  On 
the  increase  however,  is  the  switchology  associated  with  using  the  complex  displays  and 
automated  subsystems  that  are  becoming  more  and  more  common  in  modem  aircraft. 

This  increasing  complexity  means  that  motor  control  could  be  an  important  issue  for 
selection  and  training,  but  it  is  not  clear  how  this  could  be  directly  related  to  SA.  It  would  be 
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unwise,  however,  to  overlook  the  possibility  of  an  indirect  connection  between  motor  control 
abilities  and  SA.  It  is  conceivable  that  pilots  with  better  motor  control  abilities  i^uire  less 
attention  allocated  to  the  rudiments  of  flying  the  aircraft  and  controlling  the  switches.  This  might 
reduce  the  potential  for  distracting  the  pilot  away  from  maintaining  SA. 

Cognitive  Issues 

Cognitive  issues  refer  to  the  more  central  information  processing  issues  associated  with 
SA.  These  seem  likely  to  be  more  important  than  the  peripheral  processes.  In  fact,  a  recent 
review  on  SA  by  Hartman  and  Secrist  (1991)  concludes  that,  "situational  awareness  is  principally 
in  the  cognitive  domain"  (p.  1089).  Unfortunately,  the  relevant  issues  in  the  cognitive  domain  are 
very  difficult  to  categorize. 

One  of  the  main  difficulties  in  understanding  the  cognitive  issues  associated  with  piloting 
an  aircraft  involved  in  air-to-air  combat  is  the  complexity  of  the  environment  itself.  Air-to-air 
engagements  occur  within  a  longer  mission.  Numerous  goals  and  criteria  may  influence  a  pilot's 
decision-making  at  any  point  in  time  (e.g.,  reaching  a  target,  defending  yourself,  monitoring 
aircraft  status,  fuel  management,  etc.).  Accomplishing  any  goal  might  involve  multiple  tasks  (e.g., 
maneuvering  the  aircraft  while  selecting  a  weapon’s  mode  to  initiate  an  attack).  Furthermore,  any 
representation  of  the  task  environment  is  likely  to  be  very  complex.  Consider  that  in  order  to 
build  a  three-dimensional  representation  of  the  battle,  the  pilot  must  obtain  information  from  a 
two-dimensional  flat  radar  screen.  The  range  and  azimuth  information  is  usually  presented  in  a 
spatial  format  on  the  screen,  with  some  associated  information  (e.g.,  altitude,  closure  rate) 
presented  digitally.  The  pilot  will  also  communicate  via  the  radio  with  his  wingman,  and  must 
correlate  the  wingman's  interpretation  of  the  situation  with  his  own.  Further  communications  may 
occur  with  either  a  Ground  Controller  and/or  an  AW  ACS  controller  All  of  these  diverse  forms 
of  information  must  be  combined  into  a  composite  understanding  of  the  situation  that  will  guide 
action  against  a  threat  that  is  actively  trying  to  confuse  and  attack  the  pilot.  In  many  scenarios,  all 
of  this  processing  is  occurring  while  the  pilot  and  his  potential  enemy  are  closing  at  rates  of  about 
20  nm  per  minute. 

The  cognitive  abilities  a  human  operator  brings  to  performing  these  complex  tasks  are 
notable  for  their  seemingly  severe  limitations.  Attention  seems  to  suffer  when  divided  over  even 
two  fairly  simple  tasks,  and  short-term  memory  has  a  well-known  and  seemingly  quite  fixed 
capacity  of  7  plus  or  minus  2  items.  Human  operators  seem  to  be  poorly  suited  for  multi-task 
environments,  yet  they  perform  in  them  regularly.  How  can  this  be? 

Hartman  and  Secrist  (1991)  emphasize  three  phenomena  that  they  believe  are  essential  to 
creating  and  maintaining  SA  in  the  complex  air  combat  arena;  (1)  Automaticity,  (2)  Near- 
Threshold  Processing,  and  (3)  Skilled  Memory.  Automaticity'  refers  to  the  fast,  seemingly 
effortless  processing  that  produces  performance  in  some  well-trained  tasks.  Developing 
automaticity  is  primarily  a  result  of  producing  a  training  regime  that  emphasizes  the  consistent 
components  available  in  a  task  environment,  and  then  overlearning  the  detection  of  and  response 
to  these  consistent  events.  However,  Schneider  (1985)  cautions  that  this  has  implications  for 
selection.  Schneider  points  out  that  initial  performance  in  a  consistent  component  skill  is  only 
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weakly  correlated  to  final  performance  on  that  same  skill.  This  implies  that  any  selection  that  is  to 
be  done  on  components  that  will  automatized  by  the  training  should  be  done  following  some 
training. 

Hartman  and  Secrist  (1991,  p.  1087)  define  the  role  of  near-threshold  processing  as 
follows;  "The  essence  of  near-threshold  processing  is  that  individuals  can  acquire,  analyze, 
process,  and  be  affected  by  sensory  stimuli  below  the  level  of  conscious  awareness  and  that 
cognitive  and  decision  processes  can  be  modified  as  a  result."  Clearly,  an  ability  or  skill  to  do  this 
sort  of  processing  would  greatly  alleviate  the  demands  on  the  very  limited  conscious  processing 
available  to  pilots.  At  the  moment,  it  remains  to  be  determined  whether  this  is  more  usefully 
conceptualized  as  an  ability  that  should  be  measured  in  the  course  of  selection,  or  as  a  general 
skill  that  can  be  trained  m  the  student  pilot  population,  or  as  some  combination  of  the  two. 

Finally,  Hartman  and  Secrist  (1991)  identify  skilled  memory  as  a  major  contributor  to  SA. 
This  is  consistent  with  another  current  review  of  cognitive  issues  in  complex  task  performance  by 
Adams,  Tenney,  and  Pew  (1991).  Adams  et  al.  (1991)  offer  a  careful  analysis  of  the  means  by 
which  a  human  operator  overcomes  information  processing  limits  on  attention  and  memory  to 
perform  adaptively  in  complex  multi-task  environments.  Many  of  tlie  issues  discussed  are  beyond 
the  scope  of  the  present  paper,  but  the  main  thrust  is  that  developing  expertise  is  the  method  that 
operators  use  to  overcome  the  limits  of  their  cognitive  systems.  Expertise  aids  attentional 
allocation  in  that  attention  will  be  directed  to  the  information  source  that  is  relevant  to  the 
currently  most  important  goal.  Expertise  aids  memory  in  an  even  more  dramatic  fashion.  Little 
can  be  done  to  change  the  7  plus  or  minus  2  limit  on  STM,  but  with  training  the  use  of  memory 
itself  becomes  skilled.  Skilled  memory  has  been  extensively  studied  by  a  group  of  researchers  at 
Camegie-Mellon  University  (e.g.,  Ericsson  and  Straszewski,  1989).  Skilled  memory  theory 
postulates  tliree  principles:  Ihs  Meaningful  Encoding  Principle  states  that  experts  durably 
encode  task-relevant  information.  The  Retrieval  Structure  Principle  states  that  experts  possess 
memory  structures  to  support  the  facile  retrieval  of  task-relevant  information.  Finally,  the  Speed¬ 
up  Principle  states  that  with  practice  the  information  processing  associated  with  a  task  is 
performed  more  quickly  and  efficiently.  The  overall  effect  of  skilled  memory  is  that  an  individual 
who  possesses  or  has  developed  skilled  memory  in  a  domain  appears  to  have  a  functionally 
expanded  memory  through  the  efficient  exchange  of  information  from  STM  to  LTM.  This  will 
produce  impressive  performance  within  the  domain  but  will  transfer  poorly,  if  at  all,  to  any  other 
domain. 

The  fact  that  such  expertise  appears  to  be  domain-specific  presents  a  challenge  for 
personnel  selection.  Is  there  anyway  to  select  individuals  that  are  especially  prone  to  developing 
effective  skilled  memory?  No  evidence  either  way  exists.  Thus,  tests  of  basic  memory  abilities 
should  be  examined  for  their  relevance  to  the  creation  of  skilled  memory  and  new  tests  aimed  at 
examining  the  development  of  skilled  memory  in  a  simple  domain  might  be  considered. 

The  tenets  of  skilled  memoiy  theory  also  agree  with  another  general  trend  in  decision 
making  research.  Klein  (1989;  Klein  and  Klinger,  1991)  has  suggested  that  the  traditional 
theories  of  decision  making  proposed  a  level  of  information  processing  that  was  tai  too 
demanding  to  be  adaptive  in  naturalistic  real-time  environments.  As  an  alternative,  he  suggests  an 
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approach  based  on  a  Recognition-Primed  Decision  Model.  In  this  approach  the  operator's 
expertise  allows  the  operator  to  classify  the  task  as  an  example  of  some  familiar  category.  This 
classification  will  serve  to  provide  such  things  as  goals,  information  to  seek  or  monitor,  and 
courses  of  action.  This  predisposition  to  act  according  to  previous  experience  (with  suitable 
modification)  reduces  the  computational  demands  greatly  as  opposed  to  generating  and  evaluating 
a  large  selection  of  options.  Such  a  strategy  is  consistent  with  meaningful  encoding  activating  a 
course  of  action  from  a  domain  specific  retrieval  structure.  However,  Klein's  viewpoint  suggests 
that  the  main  work  done  in  real-time  decision-making  may  be  perceptual  in  nature,  thus 
suggesting  that  perceptual  tests  might  be  an  important  selection  tool.  There  is  no  evidence  to 
support  this  conjecture  at  the  moment,  but  it  is  empirically  testable. 

On  the  other  hand,  Adams  et  al.  (1991)  point  out  that  there  does  seem  to  be  evidence  that 
individuals  can  be  categorized  according  to  the  effectiveness  of  their  attentional  allocation.  For 
example,  in  a  test  of  Israeli  flight  cadets,  the  selective  attention  capabilities  as  measured  by  a 
dichotic  listening  test  were  significantly  better  for  successful  flight  cadets  than  for  those  that 
ultimately  failed  (Gopher,  1982).  Furthermore,  performance  on  this  test  was  not  completely 
correlated  with  performance  on  other  tests  in  the  selection  battery,  indicating  that  the  test  tapped 
an  ability  that  had  previously  not  been  measured.  In  an  independent  evaluation  the  test  was  used 
to  measure  the  attentional  capabilities  of  professional  bus  drivers  (Kahneman,  Ben-Ishai,  and 
Lotan,  1973).  Performance  on  the  test  was  significantly  correlated  with  the  drivers'  accident  rate. 
Tliis  implies  that  the  attentional  capabilities  remain  a  limiting  factor  in  performance  at  even  high 
levels  of  expertise. 

There  is  an  interesting  contrast  between  this  apparent  stability  of  attentional  capabilities 
observed  in  the  above  studies  and  some  research  regarding  the  training  of  attentional  control  as  a 
psychological  skill.  Gopher  and  his  colleagues  (Gopher,  1993;  Gopher,  Weil,  Baieket,  and  Caspi, 
1988;  Gopher,  Weil,  and  Seigel,  1989)  have  demonstrated  that  using  a  complex  video  game  with 
a  training  program  that  emphasizes  developing  attentional  control  skills  has  a  dramatic  effect  on 
performance  in  flight  training.  Israeli  Air  Force  cadets  that  received  the  "Space  Fortress"  video 
game  attention  training  program  had  only  half  the  attrition  from  flight  training  a  year  and  a  half 
later.  The  video  game  ti  aining  has  become  a  part  of  the  Israeli  Air  Force  training  program.  Hart 
and  Battiste  (1992)  replicated  the  efficacy  of  the  program  in  a  similar  evaluation  conducted  with 
US  Army  Helicopter  trainees  at  Fort  Rucker,  Alabama.  Not  only  did  Hart  and  Battiste  find  that 
the  "Space  Fortress"  training  aided  flight  school  performance  relative  to  trainees  with  no  video 
game  experience,  but  they  compared  the  effect  of  "Space  Fortress"  training  to  the  effect  of  a 
placebo  control  using  a  commercial  video  game  ("Apache  Strike").  The  "Space  Fortress"  group 
outperformed  both  the  no-game  group  and  the  "Apache-Strike"  group.  This  suggests  that  the 
observed  effects  are  neither  a  placebo  effect  nor  an  outcome  of  just  any  video  game  experience. 
Instead,  the  results  suggest  that  attentional  control  is  a  leamable  skill.  It  would  be  usef^ul  to 
determine  whether  the  "Space  Fortress"  effect  altered  people's  performance  in  the  dichotic 
attention  test  described  above.  Whether  this  time-sharing  effect  turns  out  to  be  an  ability  or  a 
trainable  skill  may  be  less  important  than  the  more  general  implication  that  such  time-sharing 
phenomena  are  potentially  an  important  part  of  creating  and  maintaining  SA.  Therefore, 
attentional  control  might  be  considered  a  fourth  key  cognitive  issue  to  be  considered  along  with 
the  three  suggested  by  Hartman  and  Secrist  (1991). 
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A  final  concern  that  is  not  normally  considered  "cognitive,"  but  fits  here  as  well  as 
anywhere,  is  the  stress  resistance  of  the  pilots.  High  information  loads  alone  can  be  considered 
stressors.  When  these  information  loads  are  combined  with  the  challenging  physical  demands  of 
flying  a  high  performance  aircraft,  and  the  risks  to  oneself  and  others,  the  resulting  environment  is 
fraught  with  potential  stress.  Youngling  et  al.  (1977)  conclude  that  reactions  to  stress  and 
emotional  control  are  an  important  aspect  separating  aces  from  average  combat  pilots.  At  first 
consideration  it  might  be  difficult  to  see  any  relation  between  this  and  SA.  However,  work  in 
sports  psychology  aimed  at  reducing  adverse  athlete  reactions  to  stress  tend  to  utilize  atlentional 
control  training,  typically  combining  imagery  and  relaxation  training  (Martens,  1987).  Oddly, 
when  imagery-based  techniques  have  been  studied  in  Air  Force  settings,  the  focus  has  been  on 
their  potential  motor  learning  effects  not  the  potential  attentionai  and  stress  control  benefits  (e.g., 
Prather,  1973).  Again,  research  is  needed  to  distinguish  between  stress  resistance  and  emotional 
control  as  an  individual  trait  or  a  trainable  skill.  In  cither  case,  strong  reaction  to  stress  or  lack  of 
emotional  control  is  likely  to  disrupt  attempts  to  maintain  SA. 

ImplicatiQiTis 

Implications  of  the  above  discussion  of  SA  fall  into  three  broad  categories:  selection, 
training,  and  measurement.  Each  of  these  will  be  discussed  in  turn,  but  it  is  important  to  keep  in 
mind  that  in  practice  it  would  be  unwise  to  try  to  study  these  issues  in  isolation  from  each  other 
since  they  would  be  strongly  inter-related. 

Selection  of  individuals  that  possess  abilities  beneficial  to  SA  certainly  makes  sense  if  it 
can  be  accomplished.  On  the  basis  of  the  above  discussion  it  seems  likely  that  selection  tests  of 
the  perceptual -motor  capabilities  are  the  most  straightforward  selection  contribution.  Tests  of 
visual  acuity  and  control  of  depth  of  focus  might  be  a  start.  However,  if  the  current  trend  toward 
Beyond-Visual-Range  engagements  continues,  it  is  difficult  to  see  much  of  a  contribution  from 
this  type  of  testing.  Therefore,  the  bigger  payoff  might  be  in  tests  of  perceptual  function  (i.e., 
speed  and  accuracy  of  perception,  ability  to  utilize  near-threshold  information,  integration  of 
multiple  sources  of  information  into  one  controlled  response,  etc.).  Not  only  do  such  perceptual 
tests  have  good  face  validity  for  their  value  in  human  performance,  but  the  discussion  of 
recognition-primed  decision  making  jggests  that  basic  perceptual  functioning  may  play  a  vital 
role  in  what  was  traditionally  thought  to  be  the  domain  of  higher  cognitive  functioning.  Within 
cognitive  processing  itself,  it  would  be  advisable  to  test  the  .speed,  accuracy,  and  capacity  for 
dealing  with  cognitive  processing  within  different  formats  or  codes.  Many  of  the  pilot’s  displays 
present  spatial  representations  of  the  situation  and  the  verbal  protocol  data  discussed  above 
implies  that  such  representations  are  an  important  portion  of  the  expert  pilot's  thought  processes. 
Therefore,  tests  of  the  potential  pilot's  spatial  processing  or  imagery  abilities  might  be  appropriate. 
However,  whatever  spatial  model  the  pilot  might  be  maintaii  ing  to  perform  the  task  will  have  to 
be  supplemented  by  verbal  information  from  other  pilots  or  controllers,  so  testing  verbal  abilities 
and  capacities  also  makes  sense.  Tests  of  attentionai  control  might  also  predict  the  ability  to  cope 
with  complex  dynamic  environments.  The  two  topics  discussed  above  that  might  prove 
prohibitively  difficult  to  implement  in  a  traditional  selection  battery  would  be  the  acequacy  of  the 
individual’s  domain-specific  skilled  memory.  Since  all  three  principle's  of  skilled  memory  require 
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extensive  domain-specific  training  to  develop,  they  can  only  be  tested  well  into  the  training 
program.  This  is  not  attractive  from  a  selection  point  of  view.  Also,  within  the  ethics  that  prevail 
in  the  normal  testing  environment,  it  is  unlikely  that  a  compelling  manipulation  of  the  sort  of  stress 
faced  in  the  real-world  task  will  be  achievable. 

A  preliminary  battery  of  potential  selection  tests  is  presented  in  Table  1 .  These  tasks 
represent  a  subset  of  the  tasks  used  in  the  Basic  Attributes  Tests  (BAT)  system  (Carretta,  1987) 
and  others  developed  in  the  Air  Force's  Learning  Abilities  Measurement  Program  (LAMP).  The 
tests  were  selected  to  cover  the  wide  variety  of  abilities  covered  in  the  present  paper  with  at  least 

TABLE  1. 

Proposed  Battery  of  Selection  Tests 


Test  Name 


Attribute  Tested 


Anticipation 


\VMV2 

WMV4 


Manikin 
Water  Tower 


Shepard/Metzler 

Pitch/Roll/Yaw 


Mental  Rotation 
CAMl  Fie  Mat 


Inst  Comp 
Matrices 


Scan  &  Allocate 
Time-Share  2 


Laser  Shoot  1 
Laser  Shoot  2 
Complex  Coordination 


Hartman 


Scheduling  2 
Dichotic  Listenin 


Space  Fortress 


Temporal  Processing 


Verbal  Working  Memory 


Spatial  Working  Memory 


Spatial  Orientation 


Spatial  Visualization 


Spatial  Relationships 


Spatial  Reasoning 


Time-Sharing 


Psychomotor 


Near-Threshold  Processing 


Attentional  Control 


Complex  Attentional  Control 


two  tests  for  any  individual  abilities.  The  redundancy  of  tests  for  the  different  abilities  is  essential 
to  ensure  accurate  interpretation  of  the  results.  In  addition  to  the  more  traditional  selection  tests 
the  proposed  battery  includes  a  near-threshold  processing  test  developed  by  Bryce  Hartman  and 
the  Space  Fortress  task.  Inclusion  of  these  tests  was  encouraged  by  the  previous  results  and 
analyses  reviewed  above. 

Training  issues  complement  the  discussion  of  selection  issues.  In  several  of  the  topics 
above,  it  was  unclear  whether  an  attribute  (e  g.,  control  of  depth  perception,  or  attentional 
control)  was  a  fixed  ability  or  a  trainable  generic  psychological  skill.  Training  with  a  task  like  the 
"Space  Fortress"  game  is  too  time  consuming  (about  10  hours)  to  fit  within  the  framework  of  the 
traditional  task  battery.  However,  if  such  training  can  compensate  for  attentional  control 
deficiencies  that  might  show  up  as  a  result  of  selection  testing  (perhaps  with  the  dichotic  listening 
test),  then  a  bad  attentional  control  score  might  not  have  to  be  a  basis  for  rejecting  someone  from 
a  training  program. 

Finally,  any  research  in  an  area  requires  the  ability  to  measure  outcome.  Fundamental 
research  in  generic  approaches  to  measuring  SA  has  been  underway  for  several  years,  and 
progress  has  been  made.  However,  a  generic  SA  measurement  instrument  is  prone  to  the  same 
limitation  discussed  above  with  selection  tools.  Generic  measurements  will  be  ill-suited  for 
measuring  the  domain-specific  aspects  of  a  person's  expertise.  A  potential  way  to  compensate  for 
this  is  to  develop  domain-specific  measurement  tools.  In  the  air-to-air  combat  domain  a  viable 
strategy  is  to  use  experts  to  evaluate  the  performance  of  trainees.  Instructor  ratings  have  a  long 
history  of  value  in  classifying  student  capabilities  (Youngling  et  al.,  1977).  The  expert  raters 
could  conceivably  try  to  rate  all  of  the  dimensions  discussed  above.  But,  given  that  selection  tests 
will  probably  be  well-suited  for  assessing  the  perceptual-motor  aspects  discussed  above,  the 
biggest  payoff  for  the  expert  ratings  might  be  in  the  area  of  domain-specific  knowledge  for  which 
it  is  so  difficult  to  develop  a  selection  test.  To  do  this  properly,  we  need  tools  to  extract  and 
represent  the  expert's  knowledge  about  performing  a  mission.  This  representation  can  then  be 
used  to  specify  the  dimensions  of  a  rating  scale  to  be  used  by  experts  observing  a  students 
performance.  Progress  in  this  d.rection  has  already  started:  Houck  and  his  colleagues  (Houck, 
Whitaker,  and  Kendall,  1991,  1992)  have  created  a  behavioral  taxonomy  for  air-to-air  combat  in 
F-15s.  The  taxonomy  is  based  on  mission  and  cognitive  task  analyses  of  the  defensive  counter-air 
missions  flown  by  F-15  pilots.  On  the  basis  of  extensive  interviews  of  F-IS  pilots,  Houck  et  al. 
(1991,  1992)  identified  the  critical  activities,  decisions,  and  information  requirements  of  the 
mission.  Elements  of  this  taxonomy  should  make  an  excellent  starting  point  for  developing  a 
specific  observational  rating  tool  for  evaluating  the  performance  of  pilots  in  a  air  combat 
simulation.  The  end  result  should  be  a  reliable  quantification  tool  for  assessing  a  trainee's  current 
state  of  knowledge  within  a  task  domain.  These  ratings  can  then  be  used  as  dependent  measures 
in  whatever  evaluation  might  be  conducted  or  as  an  aid  for  making  a  training  program  adaptive  to 
a  trainee's  needs.  Given  the  ability  to  put  the  trainee  in  a  stressful  environment,  expert  evaluation 
might  be  one  of  the  most  efficient  ways  to  examine  the  trainee's  grace  under  pressure. 

Consequently,  we  can  conclude  that  the  most  effective  way  to  make  progress  in  any  of 
these  domains  (selection,  training,  and  measurement)  is  to  begin  a  coordinated  program  that 
examines  all  three  issues  and  the  relationships  between  them. 
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SA:  An  Operational  Point  of  View 


Lt  Col  Eric  Vogel 
Crew  Systems  Directorate 
Crew  Technology  Division 

In  the  1970s,  crewmembers  attending  C-130  (i.e.,  the  Lockheed  C-130  Hercules  transport 
aircraA  or  "Herk")  upgrade  training  at  Little  Rock  Air  Force  Base,  Arkansas  were  shown  a  film 
entitled  "Anything,  Anywhere,  Anytime"  (1971,  AFCVBL  Film  #20981,  Norton  AFB).  The  film, 
describing  the  special  ability  of  Herk  crews  to  deliver  "anything"  (personnel,  a  variety  of  cargo) 
"anywhere"  (worldwide,  using  airdrop  or  airland  methods)  at  "anytime"  (around  the  clock,  during 
war  or  peace),  featured  the  conflict  in  Southeast  Asia  and  focused  on  the  battle  of  Khe  Sanh.  The 
viewing  served  as  an  "attention  getter"  and  motivator  for  students. 

The  film's  contents  also  bring  to  mind  several  important  aspects  of  Situational  Awareness 
(S  A).  First,  a  pilot  (or  other  crewmember)  who  possesses  good  S  A  will  strive  to  maintain  it  ~  in 
order  to  handle  "anything"  (task,  threat,  problem,  etc.),  "anywhere"  (scenarios,  locations),  and  at 
"anytime"  (time  here  can  be  thought  of  in  the  sense  of  24  hours  or  present/future) .  Secondly,  the 
crew  concept  stressed  in  the  film  highlights  the  fact  that  crewmembers  play  different  roles  in 
accomplishing  a  mission  and,  in  doing  so,  contribute  to  the  overall  crew  or  team  SA. 

The  following  discussion  of  SA  has  more  of  an  operational  than  scientific  flavor.  Airplane 
"drivers"  and  those  investigating  the  scientific  aspects  of  S  A  can  benefit  from  exposure  to  each 
other's  concerns  and  areas  of  emphasis.  SA  will  not  be  defined  nor  vrill  cognitive  aspects  be 
discussed  in  detail;  those  topics  are  covered  separately  in  this  tech  report.  The  first  section  of  this 
paper  will  comment  on  the  direction  of  current  SA  research.  Next,  an  operator's  thoughts  on  the 
major  factors  that  contribute  to  good  S  A  will  be  presented.  Finally,  through  a  description  of  a 
tactical  airdrop  scenario,  a  broader  view  of  SA  -  featuring  CRM  (Cockpit  or  Crew  Resource 
Management)  -  will  be  presented. 

Current  Research  -  A  Commentary 

Much  of  the  literature  on  S  A  is  found  in  two  main  categories  of  publications:  scientific 
journals  and  papers,  and  operational  type  flying  magazines.  This  tech  report  falls  into  the  first 
group.  A  quick  reading  of  the  keywords  reveals  that  current  scientific  writing  on  SA  focuses  on 
the  following  areas:  definition,  measures,  workload,  and  modeling.  Furthermore,  when  focusing 
on  flight  operations,  most  SA  studies  feature  air-to-air  or  air  combat  maneuver  (ACM)  scenarios. 

The  second  source  of  SA  literature  is  the  series  of  "trade"  or  operational  reviews,  such  as 
the  Mac  Flyer  (now  the  Mobility  Forum  of  Air  Mobility  Command),  TAC  Attack  (now  Combat 
Edgeof  Air  Combat  Command)  of  the  USAF,  Approach  of  the  U.S.  Navy,  and  Airline  Pilot  of 
the  Air  Line  Pilots  Association.  SA-related  articles  found  in  these  types  of  publications  are 
characterized  by:  (1)  an  emphasis  on  safety;  (2)  constant  stress  on  cockpit-,  procedural-,  and  self- 
discipline;  and  (3)  in  the  case  of  transport  aircraft,  a  highlighting  of  the  importance  of 
Cockpit/Crew  Resource  Management. 


The  primary  differences  between  these  two  groups  of  SA  literature  are  as  follows. 
Scientific  articles  normally  focus  on  a  specific  aspect  of  S A,  while  operational  type  reviews  tend 
to  take  a  broader  point  of  view.  Authors  of  scientific  articles  normally  make  their  points  by  means 
of  experiments  and  studies;  operational  articles  make  use  of  accident  report  summaries  (case 
studies)  or  "attaboys"  (citations  for  commendable  performance  during  incidents)  to  educate 
and/or  train.  Finally,  many  scientific  articles  define  SA  in  the  text;  operational  articles  often  talk 
about  the  concept  without  defining  it.  (The  lack  of  an  SA  definition  in  the  second  group  may 
follow  from  an  assumption  that  the  flying  community  has  a  general  understanding  of  SA.  Air 
Force  Manual  51-37,  the  "bible"  of  instrument  flying  for  Air  Force  pilots,  does  not  define  SA, 
even  though  good  SA  is  critical  to  instrument  flying.) 

From  a  strictly  operational  point  of  view,  SA  is  most  often  viewed  as  a  broad  concept,  or 
state  (similar,  perhaps,  to  the  "gestalt"  of  some  scientific/research  literature).  Therefore,  the 
narrow  focus  of  a  scientific  study,  by  its  specificity,  may  be  a  limitation.  Furthermore,  that  which 
is  measured  may  only  be  a  symptom,  result,  or  projection  of  SA,  but  not  a  true  measure  of  overall 
operational  SA.  This  may  be  the  case  in  a  memory  probe,  for  example.  Of  course,  this  does  not 
mean  that  scientific  or  laboratory  studies  of  SA  are  of  no  value;  in  fact,  new  discoveries 
concerning  specific  aspects  of  SA  abilities  might  aid  future  "fine  tuning"  of  operational  expertise 
and  both  individual  and  team  SA  capacity. 

Is  it  possible  to  determine  if  a  crewmember  (or  crew  "team")  has  a  good  SA  without 
defining  the  concept  and  without  "measuring"  it?  Yes,  indirectly.  At  one  end  of  an  SA  "scale," 
basic  survival  (meaning  aircrew,  wingmen  and,  if  applicable,  passengers  alive;  aircraft  intact)  is  an 
indicator  of  at  least  minimally  sufficient  SA,  for  RTB  ("return  to  base"  in  pilot  lingo)  has  been 
accomplished  and  all  will  fly  again.  Include  threats,  mechanical  problems,  weather,  unforeseen 
circumstances,  etc.,  and  that  sufficiency  of  SA  may  now  be  considered  at  least  good  SA,  because 
that  crew's  SA  has  been  challenged  and  successfully  exercised  to  overcome  problematic  situations. 
Now,  consider  those  types  of  problems  (or  challenges  to  S  A)  just  described,  but  add  mission 
accomplishment,  be  it  bombs  on  target,  air  supremacy  established,  ground  support  attained,  or 
cargo/passengers  delivered.  That  crew  which  is  aware  of  -  and  copes  with  -  all  situations,  while 
not  only  surviving  but  accomplishing  the  mission,  is  operating  in  the  realm  of  excellence  at  the 
high  end  of  the  SA  "scale." 

In  July,  1989,  a  United  Airlines  DC- 10  crashed  at  Sioux  Falls,  Iowa.  Although  1 1 1 
persons  died,  this  accident  serves  as  an  excellent  example  of  SA  at  the  high  end  of  the  scale  just 
described.  The  initial  "mission"  was  simply  to  transport  passengers  from  Denver  to  Chicago; 
however,  when  catastrophic  failure  of  the  flight  controls  occurred  (caused  by  a  separated  engine 
fan  disc  severing  hydraulic  lines),  the  mission  immediately  changed  to  one  of  simply  gaining 
control  of  the  airplane  and  landing  as  soon  as  possible  while  minimizing  deaths.  Thanks  to  superb 
SA  and  CRM  (including  outstanding  assistance  from  a  deadheading  crewmember,  air  traffic 
controllers,  and  ground  emergency  response  teams),  174  passengers  survived  in  what  could  have 
been  total  disaster. 

A  "time"  element  of  SA  is  often  mentioned  in  scientific  definitions  in  terms  of  a  pilot's 
ability  to  "project"  a  solution.  (See  definitions  of  Endsley;  Harwood,  Barnett,  &  Wickens;  and 
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Schwartz  in  the  Bibliography  section  of  this  report.)  Beginning  with  "day  one"  of  Undergraduate 
Pilot  Training,  operators  speak  of  "keeping  ahead  of  the  airplane."  This  operational  concept  of 
time  may  be  broken  into  three  divisions  as  a  technique  for  staying  aware  and  ahead;  the  present, 
the  near  future,  and  the  future  (longer  term).  The  "present"  SA  consists,  as  a  minimum,  of  the 
basics  of  position,  attitude,  altitude,  speed,  configuration,  maintenance  status,  and  environmental 
threat.  The  near  future  may  vary  in  time  from  immediately  to  several  minutes  (or  even  longer), 
depending  on  the  situation  and  the  threats  or  challenges:  "What  am  I  going  to  do  next  with  my 
machine?"  The  future  can  be  thought  of  as  a  realm  where  the  situation  for  which  the  pilot/crew  is 
mentally,  or  even  physically,  planning  and  preparing  (rigging  for  an  airdrop,  arming  weapons,  etc.) 
but  which  is  not  of  immediate  concern.  This  activity  could  be  an  approach,  landing,  bomb  drop, 
air  drop,  or  passage  through  a  threat  area.  Of  course,  these  artificial  divisions  of  time  are  fluid  in 
nature.  (For  another  example  of  SA  zones  of  interest,  see  Endsley,  1988b,  listed  in  the 
Bibliography  section.) 

What  Contributes  to  Good  SA? 

Many  factors  contribute  to  a  pilot's  or  crew's  SA.  Because  S A  is  so  complex,  some 
factors  cannot  be  quantified.  Those  factors  to  be  discussed  here  -  grouped  into  experience, 
personality,  and  mental/physical  categories  —  are  not  completely  dependent  on  superior  mental 
skills.  Nevertheless,  they  play  major  roles  in  determining  the  quality  of  SA. 

Experience.  Experience  factors  which  influence  SA  include  knowledge,  technical  skills 
and  preparation.  Obviously,  the  more  knowledge  a  pilot  has  concerning  his/her  aircraft,  weapons 
systems,  the  enemy  (specific  threat,  tactics,  capabilities,  offensive/defensive  maneuvers),  weather, 
etc.,  the  more  likely  he/she  will  be  able  to  properly  assess  a  given  situation.  Improved  technical 
skills  "  from  basic  "stick  and  rudder"  abilities  to  advanced  offensive/defensive  maneuvers  -- 
resulting  from  training  and  experience  (and,  of  course,  a  degree  of  natural  ability)  ease  mental 
workload,  allowing  for  improved  situational  awareness  and  analysis.  In  the  United  Airlines 
accident  previously  cited,  experience  and  technical  skills  (the  captain  had  been  with  United  for  33 
years;  the  deadheading  captain  was  a  check  airman)  were  critical  factors  in  averting  a  total 
disaster.  Finally,  basic  mission  preparation,  including  such  items  as  threat  analysis,  weather, 
intelligence  and  route/target  study,  covers  foreseeable  problem  situations  and  goes  a  long  way 
towards  improving  SA  and  contributing  to  the  efficient  handling  of  unforeseen  situations. 

While  performing  a  flight  evaluation  of  an  Air  Force  C-21  (Learjet)  crew,  the  author 
observed  a  young  pilot,  a  candidate  for  aircraft  commander  upgrade,  fail  a  basic  test  of  SA  while 
on  approach  to  the  busy  Los  Angeles  International  Airport.  This  pilot,  who  had  "golden  hands" 
(excellent  "stick  and  rudder"  skills)  and  good  technical  knowledge,  was  cleared  for  a  profile 
descent  (a  specific  routing  for  descent  and  arrival  to  an  airport)  by  air  traffic  control.  Initially,  he 
had  no  knowledge  of  the  procedure.  Greater  experience  and  good  mission  preparation  habits 
would  not  only  have  shown  him  where  the  procedure  was  published,  he  would  have  anticipated 
("keeping  ahead  of  the  airplane")  the  clearance.  In  this  case,  fortunately,  the  result  was 
embarrassment,  not  a  life  or  death  situation. 
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Personality.  Several  key  personality  factors  influence  the  quality  of  an  individual's  SA. 
The  pilot  who  possesses  above  average  self-discipline  is  more  likely  to  be  "in  the  books"  and  have 
a  more  professional  attitude  towards  flying  tasks.  The  pilot  who  exhibits  good  attentiveness  and 
alertness  (to  radios,  maintenance,  and  weather,  for  example),  as  well  as  a  healthy  questioning  or 
inquisitiveness  (not  to  be  confused  with  worrying)  is  working  on  excellent  SA.  Finally,  the  pilot 
who  is  always  thinking  ahead,  formulating  possible  scenarios/situations  -  and  planning  how  to 
deal  with  them  -  is  well  on  his/her  way  to  having  superior  SA. 

Two  individuals  from  the  author's  last  flying  unit  of  assignment  stand  out  as  pilots 
possessing  the  qualities  just  described  -  as  well  as  outstanding  SA.  Neither  was  satisfied  with 
basic  mission  knowledge;  in  addition  to  intimate  knowledge  of  AF  regulations,  flying  procedures, 
aircraft  systems,  and  FAA  regulations,  they  constantly  questioned  the  "how's"  and  "why's"  of 
flying,  during  flight  and  while  in  the  office.  While  on  his  first  mission  in  the  unit,  during  an 
extended  descent  to  final  approach,  one  of  the  pilots  just  described  asked  the  instructor  pilot  who 
was  flying:  "What  are  you  thinking  about?"  This  train  of  thought  --  quite  different  from  the 
common  questions  pertaining  to  systems  and  procedures  --  displayed  a  rare  level  of  analysis, 
inquisitiveness  and  aleitness. 

Mental/Phvsical  Factors.  There  are  also  several  mental/physical  factors  which  are 
related  to  the  quality  of  a  pilot's  SA.  For  example,  fatigue  (such  as  chronic  or  acute  fatigue, 
circadian  dysrhythmia,  etc.)  and  stress  (from  combat,  personal  problems,  temperature,  discomfort, 
etc.)  will  affect,  as  a  minimum,  aleitness  and  attentiveness,  key  contributors  to  good  SA.  Drugs 
and  alcohol  should  not  be  a  factor  -  if  regulations  are  followed.  However,  crewmembers  can 
follow  regulations  to  the  "T"  but  still  be  affected  by  alcohol  consumption,  either  by  short-term 
(hangover)  or  long-term  (alcoholism)  effects.  Furthermore,  self-medication,  while  forbidden, 
does  take  place,  and  the  side  affects  (dehydration,  drowsiness,  nausea,  disequilibrium,  blurred 
vision)  of  over-the-counter  medication  can  decrease  SA. 

Focus  on  Crew 

In  the  operational  world,  there  is  one  or  more  key  group  factors  influencing  the  quality  of 
SA,  in  this  case  playing  an  important  role  with  crews,  versus  individual  crewmembers.  These 
factors  are  also  the  basis  for  Crew  Resource  Management  (CRM):  communicative  ability, 
interpersonal  skills,  and  leadership.  Although  only  the  pilot  manipulates  the  flight  controls,  each 
individual  crewmember's  SA  in  his/her  specialty  (navigator,  flight  engineer,  loadmaster,  boom 
operator,  weapons  system  operator,  etc.)  is  dependent  upon  inputs  from  other  crewmembers. 
Although  the  "C"  in  CRM  serves  as  an  abbreviation  for  "cockpit"  or  "crew,"  depending  on  the 
literature,  this  author  prefers  "crew."  (This  concept  of  crew  effort  will  be  discussed  shortly  in  the 
description  of  a  tactical  airdrop  mission.)  In  fact,  there  is  an  overall  crew  (team)  SA  greater  than 
each  individual  crewmember's  knowledge  of  his/her  specialized  S A. 

Interpersonal  skills  play  an  important,  albeit  indirect,  role  in  determining  crew  SA.  A  lack 
of  skills  in  this  area  restricts  the  flow  of  information,  spirit  of  cooperation,  and  attention  to 
mission  task  necessary  for  high  level  SA  and  successful  mission  accomplishment.  A  crew  with 
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interpersonal  problems  may  function  under  routine  tasking,  but  will  be  more  likely  to  fail  when 
stressed,  subsequent  communications  falter,  and  SA,  therefore,  is  reduced. 

Leadership  is  the  third  key  area  of  CRM  which  influences  a  crew's  (and  individual 
specialty)  SA.  Although  the  "M"  in  CRM  refers  to  management,  in  fact,  proper  overall  CRM  - 
particularly  during  non-standard  operations  --  requires  leadership.  Leadership  can  vary  in  style. 

It  can  be  exercised  by  all  crewmembers,  depending  on  the  task  or  phase  of  flight,  with  certain 
{  areas  of  responsibility  -  and  the  overall  safe  conduct  and  direction  of  the  mission  --  reserved  for 

the  designated  Flight  Leader  or  Aircraft  Commander  (AC).  (Tfiis  position  would  be  the  Captain 
or  Pilot  in  Command  in  the  civilian  world  of  the  FAA.)  Here,  on  a  crew  scale,  the  leadership 
factor  -  and  the  AC's  direction  -  play  a  critical  role  in  attention  and  focus.  Just  as  the  pilot  of  a 
single  seat  aircraft  must  be  cognizant  of  many  items  making  up  SA  while  dealing  with  that  item 
which  is  currently  critical,  so  too  must  the  AC,  through  an  overall  "big  picture"  of  SA,  monitor 
the  focus  of  the  crew  and  properly  orchestrate  the  timing,  attention,  and  direction  of  a  crew. 

In  summary,  good  SA  flows  naturally  from  good  CRM.  A  crew  without  proper  CRM 
may  safely  accomplish  a  routine  mission.  Nevertheless,  when  "pressed  to  test,"  that  same  crew 
stands  a  good  chance  of  failure  due  to  incomplete  SA. 

SA  in  a  Tactical  Airdrop  Mission 

The  idea  of  "anything,  anywhere,  anytime"  as  the  basis  for  good  S  A  was  described  in  the 
beginning  of  this  paper.  Operational  aspects  of  SA  were  then  discussed.  These  two  ideas  will  be 
combined  in  the  setting  of  an  operational  mission  which  could  involve  delivering  anything 
anywhere  at  anytime:  a  C-130  airdrop. 

Although  most  discussions  of  SA  center  on  fighter  activities,  the  transport  world  provides 
a  good  example  of  how  SA  works,  especially  as  related  to  CRM.  A  C-130  crew  flying  a  low  level 
mission  to  an  airdrop  may  not  be  operating  in  the  intense  domain  of  the  "fast  movers"; 
nevertheless,  there  are  elements  of  S A  which  parallel  those  experienced  by  the  fighter  pilot.  The 
low  level  Herk  mission  can  be  every  bit  as  intense  and  merits  analysis  and  scrutiny  on  a  par  with 
that  focused  on  the  fighter  community. 

There  are  really  three  different  levels  of  SA  in  a  tactical  formation  of  C-130s.  First,  each 
individual  crewmember  has  his/her  own  area  of  expertise,  awareness,  and  concentration,  with 
some  areas  overlapping.  The  loadmaster's  SA,  for  example,  is  largely  focused  on  the  activity  in 
the  back  of  the  aircraft,  such  as  security  of  the  load  and  accomplishment  of  checklists.  He  may 
not  be  concerned  with  the  winds  over  the  drop  zone  (as  long  as  they  are  not  out  of  limits,  which 
would  necessitate  a  "no-drop"  decision  and  subsequent  actions  and  checklists),  but  the  navigator 
and  pilots  certainly  are.  The  navigator  has  an  assigned  responsibility  concerning  warnings  at 
certain  times,  yet  all  crewmembers  share  a  common  time-related  element  of  SA  -  checklist 
accomplishment. 

A  second  level  of  SA  exists  for  the  aircrew  as  a  whole.  This  concept  was  previously 
covered  under  the  discussion  of  CRM.  (As  mentioned  before,  this  author  interprets  the  "C"  as 
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Ci^w.)  The  loadmaster,  crew  chief,  drop  zone  controllers,  and  weather  personnel  are  not  in  the 
cockpit,  but  they  all  play  a  critical  role  in  mission  accomplishment.  The  AC  bears  the  major 
responsibility  for  maintenance  of  crew  SA. 

A  third  level,  or  "ring"  of  SA  exists  for  the  formation  as  a  whole.  The  formation  lead 
crew  (basically  the  pilot  and  navigator)  must  maintain  an  awareness  of  the  general  integrity  of  the 
group,  mission  progress,  and  status  of  each  aircraft  and  load.  Certainly,  each  individual  crew  is 
aware  of  their  own  aircraft  status,  but  the  leader  must  maintain  the  "big  picture." 

From  the  very  beginning  of  a  tactical  airlift  mission,  SA-related  crew  factors  play  a  part  in 
determining  success.  Even  as  a  commander  or  an  operations  officer  approves  the  schedule  of  a 
crew  and  mission,  he/she  must  consider  many  of  the  factors  cited  as  contributors  to  SA 
(experience,  personality,  CRM  leadership,  etc.)  in  order  to  bring  balance  to  the  crew  and  to 
ensure  a  safe  flight  and  accurate  drop.  It  is  common  practice  that  weaker  crewmembers  are 
teamed  with  crews  containing  stronger  fellow  crewmen. 

Planning  for  missions  can  begin  as  much  as  weeks  before  the  actual  flight  (in  the  case  of  a 
crish  or  exercise)  or  as  late  as  several  hours  before  the  airdrop  (routine  training).  During  this 
phase,  those  planning  the  mission  (usually  the  more  experienced  navigators  and  pilots)  must 
weigh  the  various  factors  of  weather  (wind,  visibility,  ceiling),  loads  (weight  limits,  drop  zone 
conditions,  drop  altitudes),  and  enemy  (real  or  simulated)  location  and  threat,  among  others.  The 
key  to  success  here  is  judgment,  brought  about  by  experience. 

During  the  briefing  phase,  most  crewmembers  (flight  engineers  and  loadmasters  are 
preparing  the  airplane  and  load)  are  together.  Here  is  an  extremely  important  opportunity  for 
establishing  the  baseline  of  individual,  team,  and  formation  SA.  The  information,  procedures,  and 
instructions  presented  are  critical  to  SA.  As  stated  earlier,  mission  preparation  plays  a  large  role 
in  superior  SA. 

During  the  flight,  the  main  focus  of  each  crewmember's  SA  is  as  follows.  The  aircraft 
commander  concentrates  on  flying  the  airplane  and,  if  serving  as  formation  lead,  on  guiding/ 
monitoring  all  aircraft  in  the  group.  The  copilot  is  responsible  for  general  checklist 
accomplishment,  navigation  (shared  with  the  navigator),  and  generally  backing  up  the  pilot  on  the 
flight  controls.  The  navigator  is  largely  responsible  for  directing  the  pilots  to  keep  the  aircraft  on 
course  and  on  time.  He  is  also  the  key  player  who  determines  precisely  where  the  load  should  be 
released.  The  flight  engineer,  seated  between  the  pilots,  monitors  the  maintenance  status  of  the 
aircraft  and  "runs"  (reads,  ensuring  proper  and  timely  responses)  the  drop  checklists.  The 
loadmaster  loads  the  aircraft,  monitors  the  load  (equipment  or  paratroopers)  for  safety  and 
security  during  the  route,  and  operates  doors  and  ensures  safe  exit  of  the  load  during  the  drop. 
Although  the  individual  areas  of  expertise  and  focus  have  just  been  described,  the  importance  of 
the  overlapping  duties  and  the  backing  up  of  each  other  -  key  aspects  of  the  CRM  concept  -- 
cannot  be  overemphasized. 

In  the  intense  environment  of  air  combat,  the  pilot  of  a  single  seat  fighter  is  faced  with  a 
complex  array  of  factors  in  maintaining  proper  SA.  During  an  airdrop  mission,  a  tactical  airlift 
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pilot  must  also  monitor  and  balance  much  information  in  order  to  maintain  good  SA.  Of  course, 
there  are  the  basics  of  speed,  altitude,  attitude,  position  (in  formation),  location  in  relation  to 
planned  track  as  well  as  specific  geographic  coordinates  (largely  a  navigator  responsibility),  fuel, 
and  maintenance  status  (generally  aided  by  the  flight  engineer),  Secondly,  there  are  items  related 
to  safety  and  security,  such  as  types  and  locations  of  tlireats,  defensive  maneuvers,  weather,  and 
terrain.  The  SA  of  these  factors  is  greatly  aided  by  preflight  preparation,  mission  planning,  and  -- 
en  route  -  good  CRM.  Finally,  keeping  in  mind  the  ultimate  goal  of  accurately  dropping  a  load 
on  target  on  time,  the  following  must  be  considered  for  proper  SA:  time  (status  in  relation  to 
preplanned  time  over  checkpoints,  completion  of  drop  checklists),  winds  (maintaining  course, 
adjusting  load  release  point),  computing  alternate  plan  (what  if  winds  are  out  of  limit?  ...  if  a 
member  of  the  formation  doesn't  drop?  ...if  maintenance  problems  or  enemy  fire  disables  an 
aircraft?).  Fortunately,  being  a  crew  aircraft,  all  crewmembers  contribute  to  the  team  SA. 

Experience  was  previously  discussed  as  a  major  enhancer  of  SA.  This  scenario  provides 
an  excellent  example  of  how  this  quality  becomes  so  important  at  a  critical  point  during  the 
mission.  Proper  position  over  the  drop  zone  is  paramount  to  a  successful  drop;  exact  alignment 
en  route  docs  not  make  a  difference  if  the  aircraft  is  off  course  and  the  load  lands  off  the  drop 
zone.  The  alignment  and  position  of  the  lead  aircraft  is  criticai;  this  is  when  the  prior  experience 
of  the  pilot  and  navigator  in  sensing  groundspeed,  wind,  and  drift  by  feel,  by  smoke  on  the 
ground,  or  by  the  "seat  of  the  pants"  can  lead  to  a  successful  drop,  regardless  of  what  the  pre¬ 
computed  air  release  point  was  calculated  to  be.  After  the  drop,  aircraft  doors  are  closed,  the 
formation  "escapes"  from  the  drop  zone,  and  post-drop  checklists  are  accomplished.  In  an  actual 
conflict,  when  a  drop  is  made  to  friendly  troops  in  enemy  territory,  the  formation  must  once  again 
transit  the  area  at  low  level  while  avoiding  and  evading  threats. 

Unfortunately,  CRM  and  SA  do  fail,  with  a  variety  of  consequences.  A  flight  engineer 
forgets  a  step  in  a  Container  Delivery  System  checklist,  the  flaps  are  not  reset  for  the  proper  angle 
for  load  release,  and  the  drop  is  long.  This  instance  was  not  only  a  mistake  on  the  part  of  the 
flight  engineer;  crew  SA  (primarily  pilot;  "What  configuration  is  the  airplane  in?")  and  CRM 
(other  crewmembers,  particularly  the  copilot,  backing  up  the  flight  engineer  on  checklist 
accomplisliment)  have  failed.  A  pilot  concentrates  on  the  runway  during  a  night  approach  to  a 
short  field  landing  and  allows  the  airspeed  to  decay  —  his  SA  fails.  Unfortunately,  the  aircraft 
stalls  and  crashes:  team  SA  and  CRM  fail  and  two  crewmembers  die.  Finally,  a  crew  fails  to 
challenge  (CRM  breakdown)  an  aircraft  commander  (experienced  C-130  pilot,  airline  captain,  and 
squadron  commander  designee)  who  decides  to  fly  low  level  on  the  spur  of  the  moment. 

Operating  in  a  partial  SA  "vacuum"  (no  previous  briefing,  no  route  study,  no  low  level  charts  on 
board),  the  air  craft  clips  a  power  line  and  all  perish. 


As  previously  noted,  many  pilot-oriented  SA  studies  concentrate  on  fighter  operations. 
However,  the  world  of  transport  aircraft,  both  military  and  civilian,  also  presents  many 
opportunities  for  studies  in  the  areas  of  individual  SA,  team  SA,  and  workload.  Both  fighter 
pilots  and  transport  crews  incorporate  many  of  the  same  elements  of  SA,  but  in  different  settings. 
A  challenged  crew  can  provide  a  wealth  of  SA  information. 


Improved  understanding  of  Situational  Awareness  is  important  to  both  the  scientific 
community  studying  SA  ccmponents  as  well  as  the  operators  flying  aircraft.  The  focus  of 
experiments  and  studies,  when  contrasted  to  the  major  operational  factors  influencing  Situational 
Awareness,  approaches  SA  from  a  different  point  of  view.  Nevertheless,  the  common  objective 
ot  mission  success  with  safety  can  be  more  readily  attained  by  a  mutual  appreciation  of  each 
group's  work  and  focus. 
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Report  of  the  Armstrong  Laboratory  Situation 
Awareness  Integration  (SAINT)  Team 
(Brieflng  Transcript) 

Dr.  Grant  R.  McMillan 
Crew  Systems  Directorate 
Human  Engineering  Division 


Bftdtgrmind 

The  present  study  was  initiated  by  a  memo  from  the  Chief  of  Staff  of  the  Air  Force. 

Figure  i  shows  that  memo  from  General  McPcak.  The  Armstrong  Laboratory's  Situation 
Awareness  Integration  (or  "SAINT")  team  was  tasked  to  respond  to  General  MePeak's  concerns. 
Lt  Col  James  Bushman  and  I  served  as  program  managers.  When  the  team  reviewed  the  memo, 
we  felt  that  he  was  asking  questions  in  three  areas:  questions  with  respect  to  the  measurement  of 
situation  awareness  (SA),  questions  with  respect  to  our  ability  to  train  SA  skills,  and,  finally, 
questions  with  respect  to  our  ability  to  select  people,  early  in  the  flying  training  process,  who  are. 
likely  to  develop  good  SA  at  some  point  down  the  line. 


CHIEF  OF  STAFF 
UNITED  STATES  AIR  FORCE 


METRICS 


SG/XO 


17  July  1991 


TRAINING 

TOOLS 


Just  what  do  we  mean  by 
“situation  awareness?” 

(I  know  it  when  I  see  it) 


Can  it  be  measured  obi 


yectively? 


Can  it  be  learned^ 

Is  it  a  sex  related  capacity? 

If  it  can  be  measured,  when  in 
the  flying  training  process 
should  we  take  measurements? 


t 

SELECTION 

TOOLS 


Menill  A.  MePeak,  General,  USAF 
Chief  of  Staff 


Figure  1.  General  MePeak’s  memo  asking  questions  about 
pilot  situation  awareness. 

In  tliis  presentation,  I  will  closely  follow  the  questions  that  he  asked  in  the  memo.  First,  I 
will  give  you  an  overviev/  of  the  program.  Then,  I  will  review  the  findings  of  the  study  in  the 
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three  areas  specified  in  the  memo:  measurement,  training,  and  selection.  Finally,  I  will  give  you  a 
summary  and  an  overview  of  our  future  plans. 

Ovfityifia 

The  conduct  of  this  study  was  truly  an  international  effort  involving  a  number  of  different 
organizations  and  operational  units.  The  primary  design  and  conduct  of  the  study  were  in  the 
United  States,  but  we  also  collected  data  as  far  away  as  Kadena  Air  Force  Base  in  Japan. 

One  of  the  first  things  our  team  did,  when  we  were  tasked  with  answering  General 
McPeak's  questions,  was  to  identify  a  working  definition.  We  adopted  the  definition  of  SA 
developed  by  an  Air  Staff  SA  working  group: 

A  pilot's  continuous  perception  of  self  and  aircraft  in  relation  to  the  dynamic 

environment  of flight,  threats,  and  mission,  and  the  ability  to  forecast,  then 

execute  tasks  based  on  that  perception. 

A  number  of  the  elements  of  this  definition  are  common  to  most  definitions  of  SA,  but 
there  are  some  things  that  are  different.  The  common  elements  include:  perception  of  the  state  of 
the  aircraft,  the  state  of  the  environment,  and  the  person's  ability  to  forecast  or  predict  into  the 
future.  But  something  that  is  unusual,  and  characteristic  of  the  view  of  operational  forces,  is  that 
SA  includes  the  ability  to  execute  tasks  in  combat. 

Obviously,  SA  is  a  veiy  complex  construct.  It  is  dilficult  to  evaluate  something  going  on 
inside  a  pilot's  head.  Since  we  cannot  measure  it  directly,  we  have  to  make  secondary 
measurements  and  draw  inferences  about  SA.  Essentially,  there  are  two  general  tecluiiques  that 
can  be  used  and,  as  you  will  see,  we  capitalized  on  both  of  them.  First,  one  can  measure  or 
observe  various  aspects  of  a  pilot's  behavior  (physical,  verbal,  etc.)  in  combat,  in  simulation,  or 
during  training  exercises  and  make  some  inferences  about  their  situation  awareness.  Alternatively, 
one  c^n  measure  fundamental  skills,  abilities,  and  characteristics  tliat  are  believed  to  be  important 
for  the  development  and  maintenance  of  SA. 

Figure  2  illustrates  the  overall  structure  of  the  study  -  the  three  main  elements  we  used  to 
answer  the  questions  asked  by  General  MePeak.  First  of  all,  we  developed  a  variety  of  rating 
scales  as  a  way  to  measure  operational  SA  out  in  the  field.  The  scale  development  effort  was 
performed  by  Dr.  Wayne  Waag  and  his  colleagues  at  the  Human  Resources  Directorate  in  Mesa 
AZ.  These  were  developed  in  several  forms:  rating  scales  that  could  be  used  by  supervisors  at  an 
operational  unit,  rating  scales  that  could  be  used  by  peers  to  rate  one  another  in  their  wings  and 
squadrons,  a  self-report  scale  that  pilots  could  use  to  evaluate  their  own  SA,  and  an  observer  form 
designed  to  evaluate  S  A  in  simulated  air  combat.  The  observer  form  was  the  primary  instrument 
that  we  used  in  our  simulator-based  testing  that's  shown  on  the  right  side  of  Figure  2. 

Essentially,  these  SA  rating  scales  were  designed  to  make  the  "I  know  it  when  I  see  it" 
process  (a  phrase  often  used  to  describe  SA)  systematic,  repeatable,  and  reliable.  I  believe  that 
we  were  successful,  as  you  will  see  from  the  data  presented  later.  We  worked  very  closely  with 
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Figure  2.  An  overview  of  the  research  program. 


pilots  from  a  variety  of  units,  including  Luke  Air  Force  Base,  to  identify  the  behaviors  they  were 
observing  when  they  say,  "This  pilot  has  SA  and  this  one  doesn’t."  Basically,  the  rating  scales 
sampled  behaviors  such  as  management  of  flight  communication,  selection  of  targets,  selection  of 
weapons,  system  management,  and  information  interpretation  and  integration.  We  extensively 
evaluated  the  scales  at  Tyndall  Air  Force  Base  to  make  sure  that  pilots  could  use  them  easily  and 
that  they  understood  what  we  were  asking  them  to  rate.  We  then  administered  the  scales  to  over 
200  pilots  in  F-15  units  at  Eglin,  Langley,  Elmendorff,  and  Kadena  Air  Force  Bases.  The  pilots  in 
each  of  those  units  had  ratings  from  30-plus  people,  including  the  peer  and  supervisor  evaluations. 

The  second  element  of  our  program  is  the  SA  Aptitude  Battery.  The  effort  to  develop, 
implement,  test,  and  run  the  aptitude  battery  portion  of  the  program  was  performed  by  Major 
David  Perry  and  his  colleagues  at  the  Human  Resources  Directorate,  Brooks  Air  Force  Base,  TX. 
The  SA  Aptitude  Battery  is  a  set  of  computer  based  tests  designed  to  measure  traits,  skills,  and 
abilities  that  are  important  for  the  development  and  maintenance  of  SA.  A  total  of  24  tests  were 
selected  based  on  our  review  of  previous  situation  awareness  research.  The  tests  measured 
elements  such  as  time  management/temporal  processing,  the  ability  to  perform  multiple  tasks  at 
once,  working  memory  capacity,  spatial  processing,  the  ability  to  recognize  briefly-presented 
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targets,  psychomotor  skills,  and  personality  variables.  Most  of  the  tests  were  already  developed, 
but  a  few  had  to  be  refined  for  our  application.  There  was  extensive  pre-testing  with  basic 
trainees  and  pilots  before  we  reduced  the  final  set  down  to  24  tests.  The  battery  was  then 
administered  to  the  same  pilots  on  whom  we  had  the  SA  Rating  Scale  data. 

The  third  element  of  the  program  focused  on  SA  measurement  in  a  demanding  and 
controlled  simulator  environment.  We  used  the  Armstrong  Laboratory  simulator  facility,  located 
principally  in  Mesa,  AZ.  Forty  pilots  were  selected  from  the  initial  set  of  over  200  to  serve  as 
experimental  pilots.  The  pilots  were  selected  to  represent  a  broad  range  of  SA  performance,  as 
indicated  by  their  squadron  ratings.  In  addition  to  the  experimental  pilots  who  were  brought  in 
two  at  a  time  from  their  units,  we  had  live  wingmen,  ground  control  intercept  (GCl)  operators, 
AW  ACS  operators  (via  a  link  to  Dr.  Schiflett's  AWACS  simulation  at  Brooks  Air  Force  Base), 
two  live  threats,  and  up  to  four  automated  threats.  The  highly-scripted  scenarios  ranged  from  lv2 
up  to  2v6.  The  experimental  pilots  were  tested  one  at  a  time  and  flew  36  engagements  over  5 
days.  We  varied  scenario  difficulty  by  manipulating  the  amount  of  SAM  activity,  communications 
jamming,  GCI  and  AWACS  support,  and  the  number  and  type  of  threats. 

In  the  simulator,  a  variety  of  dependent  variables  were  collected,  including  objective 
measures  related  to  kills,  time  to  acquire  targets,  and  eye  movement  data.  However,  the  primary 
measure  that  I'll  be  reporting  today  are  the  SA  ratings  given  by  two  ob.servers  using  the  observer 
form  of  our  SA  Rating  Scale.  The  two  observers  were  highly  trained,  ex-Air  Force  pilots  who 
were  blind  to  the  experimental  pilots'  squadron  ratings. 


'  * 

SA  COMPONENTS  IDENTIFIED 
FROM  PILOT  DEFINITIONS 

MOST  HIGHLY  RATED 
CAPABILITIES 

IN  RATING  SCALES 

Building  3-D  Composite  Picture 

Information  Integration 

Knowing  Spatial  Position  and  Geometry 

Updating  Dynamic  Siniation 

Prioritizing  Information  and  Actions 

Quality  and  Timely  Decisions 

Projecting  Situation  into  Future 

Spatial  Ability  to  Mentally  Picture 
Engagement 

Integrating  Overall  Information  from 
Multiple  Sources 

Maintaining  Track  of  Bogies  and 

Friendlies 

Use  of  Communication 

Time-Sharing  Abilities 

Adjusting  Plan  on-the-Fly 

Lookout  for  Threats  from 

Visual,  RWR,  VSD 

Figure  3.  The  key  components  of  SA  extracted  from  pilots’  definitions  of  SA 
(left),  and  the  elements  that  thQ'  rated  as  most  important  in  the 
rating  scales  (right). 
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Figure  4.  SA  rating  scale  results  from  the  in-squadron 
assessments  of  over  200  pilots. 


Measurement 


Turning  now  to  the  study  findings,  I  will  first  address  the  issue  of  measurement.  One  of 
the  first  things  that  we  did  in  testing  was  to  ask  the  pilots  for  a  personal  definition  of  SA.  What 
do  they  think  it  is?  What  are  the  key  elements?  As  a  first  step  in  using  the  rating  scales,  they  had 
to  rate  the  importance  of  each  of  the  rating  scale  elements.  Figure  3  compares  key  S  A 
components  extracted  from  the  pilots'  definitions  with  the  elements  that  they  rated  as  most 
important  in  our  rating  scales.  I  think  you  will  see  a  consistent  picture  there  -  the  elements  of 
building  a  3-D  composite  picture  of  the  battle,  information  integration,  and  so  on  down  the  list. 
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Clearly,  there  was  agreement  between  what  the  pilots  were  stating  in  their  definitions  of  SA  and 
the  elements  included  in  our  rating  scales. 

We  believe  that  we  can  answer  a  strong  'yes'  to  the  Chiefs  question  about  whether  or  not 
SA  can  be  measured.  We  combined  the  peer  and  supervisor  SA  ratings  and  did  a  principal 
components  analysis  on  them.  Using  the  first  principal  component  (which  accounted  for  92%  of 
the  variance),  we  observed  a  good  distribution  of  squadron  S  A  scores  (Figure  4.).  Obviously 
there  is  a  spread  of  ability  here.  One  of  the  things  that  pleasantly  surprised  us,  shown  in  the  figure 
labeled  "High  Agreement,"  was  the  consistency  between  the  supervisor  and  peer  ratings  of  the 


IDENTIFIED  INDICATORS 
OF  POOR  SA 

•  Reduced  Communication 

•  Reduced  Maneuvering 

•  Defensive  Reaction 

•  Channelized  Attention 

•  Poor  Pre-Merge  Kill  Ratio 

•  Lost  Awareness  of  Objectives 


IDENTIFIED  CRITICAL 
SA  ABILITIES 

•  Leadership  and  Resource 
Management 

•  Decision  Making 

•  Communication 

•  Tactics  Knowledge 

•  Weapons  Knowledge 


Figure  3.  Relationships  between  the  in-squadron  SA  rating  scale 
assessments  and  expert  observer  ratings  of  SA 
during  simulated  air  combat. 
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same  pilots.  As  you  can  see,  those  correlations  are  quite  high  -  all  in  the  .9  region.  They 
definitely  agreed  on  who  the  top  and  bottom  pilots  were.  The  correlations  between  the  self- 
reported  SA  and  the  peer/supervisor  ratings  (not  shown)  were  much  lower,  in  the  .4  to  .5  range. 

Another  thing  you  will  notice  in  the  figure  is  that  pilots  don't  make  a  strong  distinction 
between  fighter  ability  and  SA  ability.  As  far  as  they’re  concerned,  those  two  things  are  tightly 
linked  and  if  a  pilot's  got  one,  they've  got  the  other.  The  third  tiling  we  found  was  that  there  was 
no  significant  difference  in  the  average  SA  scores  of  the  squadrons  in  which  we  made  these 
measurements;  one  F-15  squadron  wasn't  significantly  higher  in  terms  of  SA  scores  than  the 
others.  This  is  illustrated  in  the  bottom  part  of  Figure  4,  and  is  another  indication  that  the  raters 
were  using  these  scales  in  a  similar  fashion. 

We  also  were  successful  in  measuring  SA  in  the  simulator  environment.  Figure  S  shows 
some  of  these  findings.  We  obser/ed  a  good  correlation  between  the  squadron  and  simulation  SA 
scores.  While  we  did  not  expect  a  ne>fect  relationship,  the  correlation  (r  =  .56)  implies  that  we 
are  measuring  some  common  eic.,  ..  lere.  We  also  found  that  highly  trained  observers  are  a 
very  effective  way  to  make  this  type  of  measurement.  When  measuring  a  complex  construct  like 
SA,  they  can  look  past  simulato"  deficiencies,  they  can  identify  "game-playing,"  and  they  can 
evaluate  performance  in  the  simt  for  environment  in  a  way  that  "objective"  measures  often  can't. 
One  of  the  things  that  they  picked  ii>i  consistently  was  indicators  of  poor  SA:  reduced 
communication,  red'  ;ced  maneuvering,  and  so  on.  They  also  reported  key  characteristics  of  pilots 
who  showed  ’  igh  S/ 


Figure  6.  One  pilot’s  dramatic  shift  in  SA  that  was  apparently  caused 
by  a  family  crisis. 
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Figure  6  presents  some  anecdotal  data  that  has  generated  a  great  deal  of  interest.  One  curve 
in  Figure  6  shows  the  average  SA  score  for  the  40  pilots  who  participated  in  the  simulation  phase  of 
this  study.  It  has  a  slight  downward  trend  since  the  scenarios  became  more  difficult  as  the  week 
progressed.  The  other  line  shows  the  data  from  one  pilot.  He  was  a  top  performer;  a  fighter 
weapons  school  graduate  who  had  high  ratings  in  his  squadron.  On  the  initial  scenarios,  he  performed 
beautifully.  Then,  on  Day  3  (Wednesday),  a  marked  drop  occurred  and  he  performed  more  poorly 
than  the  average  of  the  other  pilots.  On  Day  4,  his  performance  came  back  up  to  well  above  average. 
When  he  was  being  debriefed,  he  told  us  that  he  received  a  call  Wednesday  morning  that  his  18- 
month  old  son  was  going  into  the  hospital  for  emergency  surgery.  On  Wednesday  evening,  he  got  a 
call  that  everything  was  fine.  People  often  ask  the  question,  "Are  family  crises  going  to  affect  SA  out 
in  the  field?"  These  results  suggest  that  this  could  be  the  case.  We  were  equally  interested  in  the  fact 
that  our  observers  picked  it  up.  They  were  not  overwhelmed  by  the  halo  effect  of  his  performance  on 
the  first  two  days.  They  observed  this  marked  drop  in  SA.  even  though  they  had  no  idea  why  it 
happened. 

SA  SCORES  RELATED  TO: 

*  Flight  Qtialiflcaiion 

*  Flight  Hours 

*  Fighter  Weapons  School 

*  Exercise  Participation 

*  MACAIR  Participation 


INCREASES  WITH  QUALIFICATION 


T.  140 
>30 


FUCHT  WALinCATION 


Figure  7.  Impact  of  various  training  experiences  on 
SA  capabilities. 
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Before  discussing  the  training  implications  of  this  study,  I  must  point  out  that  these 
implications  are  based  on  correlational  analyses.  Because  of  time  constraints,  we  could  not  run  a 
rigorous  learning  or  transfer-of-training  study.  In  Figure  7,  we  show  the  relationship  between 
certain  training  variables,  training  experiences,  and  the  SA  scores  that  were  measured  out  in  the 
squadron.  We  found  that  the  squadron  SA  scores  were  statistically  related  to  the  flight 
qualification  level,  the  number  of  flying  hours,  whether  or  not  they'd  been  to  fighter  weapons 
school,  participation  in  exercises  such  as  Red  Flag,  Green  Flag,  Maple  Flag,  and  even  whether  or 
not  they  had  participated  in  other  simulation  exercises.  The  middle  and  bottom  figures  show  the 
specific  relationships  between  SA  scores  and  flight  qualification,  and  SA  scores  and  flight  hours. 
While  no  cause-and-effect  relationships  can  be  determined  from  these  findings,  the  consistent 
pattern  is  noteworthy.  Training  experiences  designed  to  enhance  pilots’  SA  are  associated  with 
increased  SA. 


SA  SCORES  INCREASE  IN  ARMSTRONG  SIM 


Q  FIRST  ENGAGEMENT 
■  LAST  ENGAGEMENT 


PILOTS  JUDGED  SIM  BENEHCIAL 


EXTREMELY 

BENEnCIAL 

HIGHLY 

BENEnCUL 

SOMEWHAT 

BENEnClAL 

BENEFICUL 


NOT 

BENEFICUL 


5 


LEAD  LEAD 


Figure  8.  Impact  of  pilot  practice  on  expert  observer  ratings  of  SA  (top)  and  pilot 
assessments  of  benefit  of  this  type  of  training  experience  (bottom). 
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More  direct  evidence  of  SA  learning  can  be  extracted  from  the  simulator  data.  For 
example,  in  the  Armstrong  Laboratory  simulator  the  pilots  went  through  a  total  of  36  different 
engagements.  The  upper  part  of  Figure  8  shows  the  SA  score  that  they  got  on  their  first  2v2 
defensive  counter  air  sortie  versus  their  last  one,  and  their  first  2v4  offensive  counter  air  and  their 
last  one.  In  both  cases,  the  SA  scores  improved.  We  also  asked  the  pilots  to  evaluate  the  training 
benefit  of  interactive  multi-ship  simulation.  The  pilots  overwhelmingly  rated  the  Armstrong 
Laboratory  simulation  experience  as  highly  or  extremely  beneficial  for  the  development  of  SA 
skills.  We  had  them  rate  the  benefit  for  a  wingman,  a  2-ship  lead  or  a  4-ship  lead,  and  they  felt 
that  the  simulator  would  be  highly  or  extremely  beneficial  for  all  those  qualification  levels. 

Selection 

Our  findings  with  respect  to  selection  do  not  permit  as  clear  an  answer  as  we  were  able  to 
offer  in  the  measurement  and  training  areas.  We  found  that  three  abilities  were  related  to 
(predicted)  SA  scores  in  the  squadron:  working  memory  capacity,  multi-tasking  ability,  and 
spatial  processing.  We  also  found  that  the  current  Air  Force  selection  system  includes  tests  of 
these  abilities  as  part  of  the  Pilot  Candidate  Selection  Model  (PCSM).  PCSM  scores  are  one 
factor  in  selecting  pilots  for  Undergraduate  Pilot  Training  (UPT).  Because  PCSM  has  only 
recently  been  fielded,  we  don't  know  if  PCSM  scores  will  predict  SA  in  mission-ready  fighter 
pilots.  We  do  know,  however,  that  PCSM  scores  predict  performance  and  attrition  in  UPT 
(Figure  9).  It  will  take  about  5-8  years  until  we  have  PCSM  data  on  enough  mission-ready  pilots 
to  determine  if  it  is  a  useful  SA  selection  tool.  Nevertheless,  our  findings  suggest  that  the  current 
selection  system  is  measuring  the  right  abilities. 


ABILITIES  TESTS  PREDICT  UPT  PERFORMANCE 


PCSM  SCORE 

■  ATTRITION  DUE  TO  FLYING  TRAINING  DEFIOENCY 
□  GRADUATION  IN  UPPER  50%  OF  CLASS 


Figure  9.  The  relationship  between  Pilot  Candidate  Selection  Model  (PCSM)  scores 
and  pilot  trainee  success  in  Undergraduate  Pilot  Training  (UPT). 
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General  McPeak  also  asked  the  question,  "Is  there  any  evidence  that  SA  skills  are  gender 
related?"  Since  we  were  only  evaluating  F-15  pilots,  we  could  not  address  this  issue  directly.  As 
an  indirect  measure,  we  looked  in  the  literature  for  evidence  of  male/female  differences  on  the 
three  ability  tests  that  were  correlated  with  the  SA  scores:  working  memory  capacity,  multi¬ 
tasking  ability,  and  spatial  processing.  We  found  no  reported  male/female  differences  on  the 
particular  tests  or  attributes  that  were  significant  predictors  of  SA  in  our  study. 

Cgnglufiifins 

In  summary,  our  findings  suggest  that  SA  can  be  measured  using  rating  scales  in  an 
operational  environment  and  using  trained  observers  in  a  controlled-exercise  environment .  Our 
data  suggests  that  the  flying  training  process  does  enhance  SA,  and  that  it  grows  with  a  wide 
variety  of  operational  experiences.  Can  we  select  for  SA?  A  solid  answer  will  require 
longitudinal  tracking  of  pilots  in  the  current  selection  and  training  pipeline,  but  we  can  use  the 
data  that  we  have  collected  to  further  refine  the  current  process. 

In  the  future,  we  plan  to  broaden  our  wcrk  in  several  areas.  We  will  evaluate  multi-ship 
simulation  and  cognitive  skills  training  using  controlled  transfer-of-training  designs.  We  will 
expand  our  SA  research  to  other  areas  -  other  aircraft,  other  jobs,  team  SA,  and  so  forth.  We 
plan  to  track  pilots  through  the  operational  process  to  see  if  tools  such  as  PCSM  can  improve 
selection.  Finally,  we  should  remember  that  improved  cockpit  design  has  the  potential  to  increase 
SA  as  much  as  training  and  selection.  Thus,  we  plan  to  develop  a  set  of  measurement  tools  to 
assess  the  S  A  benefits  of  alternative  cockpit  designs. 
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1,  Adam,  E.C.  (1993).  Fighter  cockpits  of  tho  future,  [n  Proceedings  of  the 

AiA A/IEEE  12th  Digital  Avionics  Systems  Conference  (pp.  318-323).  New  York: 
Institute  of  Electrical  and  Electronics  Engineers. 

This  paper  presents  Adam's  views  of  the  relationship  between  situation  awareness  and  fighter  cockpit 
design.  Adam  reviews  the  history  of  cockpit  development,  identities  potential  limitations  of  current 
cockpit  designs,  and  outlines  potential  future  designs  to  improve  the  fighter  cockpit.  Historically, 
Adam  suggests  that  we  have  been  through  two  generations  of  cockpit  designs;  The  first  generation 
was  characterized  by  (p.  318),  "round  dials  to  help  the  pilot  keep  the  airplane  flying  right  side  up."  The 
second  generation  of  cockpit  designs  introduced  multifunction  displays  and  head-up  displays  (HUDs). 
Current  fighter  aircraft  are  examples  of  second-generation  cockpits.  Adam  suggests  that  the  current 
designs  severely  constrain  fighter  pilot  performance  for  two  reasons:  (1)  too  much  time  is  required  to 
"fiddle"  with  aircraft  systems  rather  than  executing  tactics,  and  (2)  only  20%  to  30%  of  the  display 
area  in  the  cockpit  can  be  used  for  presenting  useful  combat  information.  Previously,  the  limited 
amount  of  reliable  tactical  information  that  was  available  within  the  cockpit  could  be  presented  and 
used  with  the  above  constraints.  However,  Adam  believes  that  modem  technologies,  such  as 
improving  on-board  sensors  and  data  links,  will  profoundly  increase  the  amount  of  data  that  must  be 
presented  to  the  pilot.  It  is  suggested  that  this  will  cause  major  problems  for  the  pilot  as  it  is  presented 
in  little  pieces  that  must  be  cognitively  "fused"  by  the  pilot.  Adam  then  interprets  potential 
improvements  to  cockpit  designs  within  a  SA  framework.  The  definition  issue  is  addressed  briefly  (p. 
319):  "So,  what  is  SA,  what  is  it  all  about?  It's  simply  KNOWING  WHAT’S  GOING  ON  SO  YOU 
CAN  FIGURE  OUT  WHAT  TO  DO! "  Adam  subdivides  overall  SA  into  two  main  components: 
Global  SA  (the  Big  Picture),  and  Tactical  SA  (The  Little  Picture),  In  the  near  tenn,  the  tactical  SA 
solution  is  seen  to  be  helmet-mounted  displays  (HMDs).  Global  SA  can  be  addressed  by  increasing 
the  cockpit  display  area  and  fusing  the  data  into  a  comprehensive  picture  on  a  Big  Picture  display. 
Several  possible  developmental  steps  towards  developing  such  future  cockpits  are  described. 

KEYWORDS;  air-to-air  mission,  air-to-ground  mission,  aviation  (military),  big  picture 

displays,  data  fusion,  data  link,  definition,  display  design,  helmet-mounted 
displays  (HMDs),  pilot  aiding 


2.  Adam,  E.C.  (1994).  Tactical  cockpits  -  The  coming  revolution.  In  R.D.  Gilson, 
D.J.  Garland,  and  J.M.  Koonce  (Eds.),  Situational  awareness  in  complex  systems: 
Proceedings  of  a  CAHFA  Conference  (pp.  101-1 10).  Daytona  Beach,  FL: 
Embry-Riddle  Aeronautical  University  Press. 

This  article  essentially  presents  the  same  definition  and  interpretations  discussed  in  the  Adam  (1993) 
paper. 

KEYWORDS:  air-to-air  mission,  air-to-ground  mission,  aviation  (military),  big  picture 

displays,  data  fusion,  data  link,  definition,  display  design,  helmet-mounted 
displays  (HMDs),  pilot  aiding 
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3.  Adams,  M.J.,  and  Pew,  R.W.  (1990).  Situational  awareness  in  the  commercial 
aircraft  cockpit:  A  cognitive  perspective.  In  Proceedings  of  the  lEEE/AIAA/NASA 
9th  Digital  Avionics  Systems  Conference  (pp.  519-524).  New  York:  Institute  of 
Electrical  and  Electronics  Engineers. 

This  paper  is  primarily  concerned  with  generating  a  theoretical  background  for  understanding  SA 
within  the  commerciaJ  aircraft  cockpit.  Situation  awareness  (SA)  is  defined  (p.  519)  as  the  crew's 
"moment  by  moment  ability  to  monitor  and  understand  the  state  of  the  aircraft,  its  systems,  and  its 
environment."  SA  is  seen  as  being  intimately  involved  with  mental  workload.  In  fact,  the  authors 
contend  (p.  520);  "From  our  perspective,  achieving  situational  awareness  is  but  a  special  class  of 
mental  workload."  In  the  discussion  of  the  cognitive  processes  involved  in  maintaining  SA,  the  authors 
emphasize  the  properties  of  the  human  memory  system  that  allow  it  to  contend  with  demanding 
real-time,  cognitively  complex  tasks.  Mental  models,  the  organization  of  long-term  memory,  the  nature 
of  expertise,  the  integration  of  new  information  into  an  on-going  process,  and  attentional  limitations 
are  among  the  issues  that  are  discussed. 

KEYWORDS:  attention,  aviation  (civilian),  cognitive  processes,  definition,  expertise, 
mental  models,  workload 


4.  Adams,  M.J.,  Tenney,  Y.J.,  and  Pew,  R.W.  (1991,  December).  Strategic 

vi/orkload  and  the  cognitive  management  of  advanced  multi-task  systems  (CSERIAC 
State  of  the  Art  Report  91-6).  Wright-Patterson  Air  Force  Base,  OH:  Crew  System 
Ergonomics  Information  Analysis  Center. 

The  main  thrust  of  this  report  concerns  a  reappraisal  of  mental  workload.  The  authors  argue  that  the 
dominant  view  of  workload  as  the  demand  and  allocation  of  mental  resources  is  too  passive.  This 
passivity  is  seen  as  especially  inadequate  when  attempting  to  explain  human  performance  in  a  dynamic 
multi-task  environment  with  advanced  automation  aids.  As  an  alternative,  they  present  a  theoretical 
framework  that  combines  active  theories  of  perception  (primarily  inspired  by  Neisser),  connectionist 
theories  from  the  Parallel  Distributed  Processing  (PDP)  school,  and  schema  theories  of  text 
comprehension.  This  framework  includes  discussion  of  the  interplay  of  information  in  the  explicit 
focus  of  short-term  memory  along  with  associated  information  that  is  relevant  to  the  ongoing 
processing,  but  outside  awareness  in  an  implicit  focus.  The  issues  of  awareness  in  general,  and 
situation  awareness  in  specific,  recur  frequently  throughout  the  manuscript.  Many  of  the  concepts 
discussed  in  this  report  are  similar  to  those  reviewed  by  Tenney  et  al.  (1992)  in  their  discussion  of 
situation  awareness  in  commercial  aviation. 

KEYWORDS:  cognitive  processes,  expertise,  review  paper,  schema,  workload 
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5.  Alkov,  R.A.  (1994,  April).  Enhancing  safety  with  aircrew  coordination  training. 
Ergonomics  in  Design,  pp.  13-18. 

This  article  is  primarily  a  review  of  the  rationale  for  applying  an  aircrew  coordination  training  program 
to  U.S.  Navy  and  Marine  Corps  helicopter  and  fighter  squadrons.  The  author  reviews  accident 
statistics  that  demonstrate  that  human  error  is  a  major  contributor  to  U.S.  Navy  and  Marine  Corps 
helicopter  and  fighter  mishaps.  The  author  contends  that  poor  pilot  coordination,  faulty  pilot 
judgment,  and  loss  of  situational  awareness  are  important  contributors  to  many  of  these  mishaps. 
Among  the  causes  of  loss  of  situation  awareness  the  author  lists  distraction,  ambiguous  information, 
work  overload,  and  poor  communication.  In  particular,  the  author  emphasizes  the  potentially  damaging 
effects  of  too  much  workload  on  SA  (p.  16):  "If  a  pilot  has  to  use  large  amounts  of  working  memory 
to  fly  the  aircraft,  those  resources  can't  be  used  to  scan  the  environment  for  information  needed  to 
maintain  situational  awareness."  As  a  means  to  alleviate  some  of  the  problems  that  can  cause  aircraft 
mishaps,  a  program  to  train  aircrew  coordination  is  advocated.  In  tests  of  the  program,  participants 
have  reported  that  they  feel  it  was  beneficial  and  accident  statistics  following  the  program  showed  a 
marked  decrease  in  the  mishap  rates  due  to  aircrew  error. 

KEYWORDS:  aviation  (military),  cockpit  resource  management  (CRM),  helicopter,  pilot 
error,  team  decision  making,  team  SA,  training,  workioad 


(5.  Anderton,  D.A.,  Soras,  C.,  and  Lewis,  W.R.  (1988,  April  11).  Market  Supplement: 
Cockpit  Technology.  Aviation  Week  &  Space  Technology,  725(15),  57-98. 

This  market  supplement  contains  an  introduction  and  S  articles.  SA  is  prominent  in  the  introduction 
and  in  the  first  two  articles.  In  the  introduction,  improving  SA  is  identified  as  a  major  goal  for  new 
cockpit  technology  development  in  both  military  and  commercial  aviation.  The  first  article,  Revolution 
in  the  Cockpit,  emphasizes  the  emergence  of  revolutionary  new'  technology  in  the  military  cockpit  and 
the  effects  on  SA.  Among  the  topics  discussed  are  "big  picture"  displays,  helmet-mounted  displays, 
and  advanced  automation  in  avionics.  The  second  article,  Automating  the  Two  Crew  Flight  Deck, 
discusses  the  development  of  the  "glass  cockpit"  for  commercial  aviation.  These  systems  are  promoted 
as  a  means  to  increase  SA  while  reducing  workload. 

KEYWORDS:  aviation  (civilian),  aviation  (military),  avionics,  big  picture  displays,  display 
design,  helmet-  mounted  displays  (HMDs),  pilot  aiding 


53 


7.  Andre,  A.D.,  Wickens,  C.D.,  Moorman,  L.,  and  Boscheili,  M.M.  (1991).  Display 
formatting  techniques  for  Improving  situation  awareness  In  the  aircraft  cockpit. 

The  International  Journal  of  Aviation  Psychology,  1,  205-218. 

The  influence  of  display  perspective,  frame  of  reference,  and  color  on  situation  awareness  in  aircraft 
navigation  was  assessed.  The  authors  expressed  doubts  about  the  appropriateness  of  subjective  recall 
procedures  for  situation  awareness  assessment.  Regardless  of  whether  the  situation  awareness  probes 
were  presented  during  the  actual  performance  of  the  task  or  in  a  retrospective  setting,  the  issue  of 
developing  an  appropriate  method  for  eliciting  subjective  recall  of  the  relevant  aspects  of  the  situation 
was  considered  daunting.  As  the  authors  put  it  (p.  209);  "Given  the  inherent  ambiguity  of  situation 
awareness  as  an  operational  construct,  there  can  be  any  number  of  possible  'details'  relevant  to  the 
mission.  Hence,  the  validity  of  each  recall  procedure  is  dependent  more  on  the  structure  of  the  subjects' 
report  than  on  the  efficacy  of  the  display  formats  in  question."  As  an  alternative  to 
memory-probe-ba.sed  situation  awareness  metrics,  the  authors  developed  performance  tests  based  on 
how  adaptively  the  subjects  achieved  appropriate  orientation  to  the  next  waypoint  in  the  simulated  task 
after  a  disruption  and  how  often  they  controlled  multiple  axes  of  the  aircraft.  The  results  indicated  that 
3-D  perspective  as  presented  in  this  experiment  did  not  improve  overall  navigation  performance,  but 
did  support  better  multi-axis  control  strategies.  Also,  the  nontraditional,  outside-in  reference  frame 
was  superior  to  the  traditional,  inside-out  display.  Overall,  the  experiment  provides  an  example  of 
using  the  situation  awareness  concept  to  help  guide  a  display  evaluation.  Although  the  specific 
measures  used  in  the  research  will  probably  net  generalize  to  other  evaluations,  the  general  approach 
might  be  useful  in  other  settings. 

KEYWORDS;  display  design,  measurement,  memory  probe  measures,  performance-based 
measures 


8.  Arbak,  C.J.,  Schwartz,  N.,  and  Kuperman,  G.  (1987).  Evaluating  the  panoramic 
cockpit  controls  and  displays  system.  In  Proceedings  of  the  Fourth  International 
Symposium  on  Aviation  Psychology  (pp,  30-36).  Columbus:  The  Ohio  State 
University. 

The  authors  argue  that  the  iterative  addition  of  new  systems  to  the  cockpit  over  the  years  has  created  a 
situation  awareness  problem  for  the  pilot  Such  systems  sometime  display  the  same  information  in 
different  formats  and  require  the  pilot  to  perform  the  difficult  job  of  information  fusion.  The 
Panoramic  Cockpit  Controls  and  Displays  System  (PCCADS)  was  developed  as  a  means  to  help  future 
pilots.  PCCADS  combined  a  large  "big  picture"  display  system  with  innovative  controls  technology. 
The  goal  was  to  increase  performance  and  situation  awareness  while  reducing  workload.  The  authors 
discussed  some  of  the  complex  ways  that  performance,  situation  awareness  and  workload  might 
interact.  For  a  planned  evaluation  of  PCCADS  the  authors  had  developed  a  subjective  situation 
awareness  scale.  The  scale  was  based  on  a  Tactical  Air  Command  definition  of  situation  awareness  and 
consisted  of  six  questions  about  situation  awareness,  each  of  which  was  answered  by  selecting  a 
response  from  a  seven  point  scale.  The  authors  reported  that  the  scale  appeared  to  be  sensitive  to  a 
manipulation  of  situation  awareness  in  a  preliminary  study. 

KEYWORDS;  aviation  (military),  big  picture  displays,  display  design,  measurement,  pilot 
aiding,  subjective  measures,  workload 
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9.  Aretz,  A.J.  (1988).  A  model  of  electronic  map  interpretation.  In  Proceedings*:  of 
the  Human  Factors  Society  32nd  Annuai  Meeting  (Volume  1,  pp.  130-134).  Santa 
Monica,  CA:  Human  Factors  Society. 

The  author  argues  that  one  component  of  situation  awareness  is  navigation  awareness  (Am  I  where  I 
think  I  am?).  Electronic  map  displays  are  being  developed  for  the  cockpit  that  are  intended  to  improve 
navigational  awareness.  The  author  proposed  and  tested  a  model  of  the  cognitive  processes  that  would 
be  needed  to  interpret  an  electronic  map  display.  Complexity  of  the  display  was  found  to  have  a 
significant  effect;  but,  somewhat  surprisingly,  the  effect  of  rotation  of  the  display  was  found  to  be 
minor. 

KEYWORDS:  aviation  (civilian),  cognitive  processes,  definition,  display  design, 
measurement,  performance-based  measures 


10.  Aretz,  A.J.  (1991).  The  design  of  electronic  map  displays.  Human  Factors,  33, 
85-101. 

The  paper  does  not  discuss  the  overall  concept  of  SA,  but  discusses  navigational  awareness  which  can 
be  considered  to  be  one  aspect  of  SA.  The  author  presents  a  theoretical  framework  in  which 
navigational  awareness  is  viewed  as  (p.  85),  "the  maintenance  of  a  cognitive  coupling  between  two 
reference  frames  (RFs)  that  correspond  to  the  map  and  the  forward  view  of  the  world."  The  paper 
discusses  the  cognitive  issues  involved  in  using  maps  to  aid  navigational  awareness,  possible 
alternative  map  display  formats,  and  the  concept  of  "visual  momentum"  as  a  display  guideline 
applicable  to  map  display  design.  The  results  of  an  experiment  are  interpreted  as  supporting  the 
notions  that  navigational  awareness  involves  cognitively  aligning  two  frames  of  reference,  that  this 
cognitive  activity  competes  with  other  cognitive  activities  for  limited  resources,  and  that  the  visual 
momentum  concept  is  a  useful  display  design  tool. 

KEYWORDS:  attention,  cognitive  processes,  design  guidelines,  display  design, 

measurement,  multiple  resource  theory,  performance-based  measures,  pilot 
aiding,  subjective  measures,  workload 
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II.  Ballard,  D.,  and  Owsley,  L.  (1991).  Artificial  intelligence  in  the  helicopter  cockpit  of 
the  future.  In  Proceedings  of  the  lEEE/AiAA  10th  Digital  Avionics  Systems 
Conference  (pp.  125-130).  New  York:  Institute  of  Electrical  and  Electronics 
Engineers. 

This  article  discusses  the  architecture  and  design  issues  for  an  artificially  intelligent  Situation 
Assessment  device  to  be  used  in  military  helicopters.  The  Situation  Assessment  device  is  based  on 
technology  developed  by  the  DARPA  (Defense  Advanced  Research  Projects  Agency)  and  the  US  Air 
Force  for  the  Pilot's  Associate  (PA)  program  and  is  intended  to  support  the  US  Army's  Rotorcrafl 
Pilot's  Associate  (RPA)  Advanced  Technology  Transition  Demonstration  (ATTD)  Program.  The 
Situation  Assessment  device  is  on  part  of  a  planned  cognitive  decision  aiding  technology  that  will 
assist  the  pilot  in  formulating  his/her  situation  awareness  by  (p.  126),  "recognizing  hostile  and 
unknown  threats  and  prioritizing  them  with  respect  to  threat  lethality."  The  authors  adopt  the 
following  definition  of  situational  awareness  (P.  126);  "Situational  awareness  is  knowing: 

-  Where  friendly  ground  and  airborne  forces  are  and  what  they  are  doing; 

-  Where  threats  are  and  what  they  are  doing; 

-  What  ownfiight  knows  and  options  for  offense/defense; 

-  What  other  flights  know  and  their  intentions,  and  finally  and  perhaps  most  importantly, 

-  What  part  of  the  above  is  not  known  or  is  missing." 

KEYWORDS:  aviation  (military),  avionics,  decision  aiding,  definition,  helicopter,  pilot 
aiding 


12.  Balias,  J.A.,  Heitmeyer,  C.L.,  and  Perez,  M.A.  (1991).  Interface  styles  for  adaptive 
automation.  In  Proceedings  of  the  Sixth  International  Symposium  on  Aviation 
Psychology  (Volume  1,  pp.  108-113).  Columbus:  The  Ohio  State  University. 

This  research  investigated  the  use  of  graphical  direct  manipulation  displays  to  reduce  the  possibility 
that  automation  would  degrade  SA.  The  subjects  performed  a  laboratory  dual  task  combining  a 
tracking  task  and  a  target  classification  task.  The  difficulty  of  the  tracking  varied  over  time.  As  the 
tracking  became  difficult,  the  target  classification  task  would  adaptively  shift  to  an  automated  state  in 
which  the  subjects  merely  confirmed  the  classification.  The  display  for  the  target  automation  task  was 
either  graphical  (with  icons)  or  tabular,  and  responses  to  the  target  identification  task  were  either 
through  a  keypad  or  via  a  touch  screen.  The  hypothesis  was  that  a  direct  manipulation  interface  (e.g., 
the  combination  of  graphical  presentation  and  touch  screen  response)  would  allow  better  and  more 
efficient  maintenance  of  SA  than  would  a  non-direct  manipulation  display  (e  g.,  tabular  display  with 
keypad  response).  Although  the  data  collection  had  not  been  completed  there  was  a  non-significant 
trend  in  the  performance  data  that  suggested  that  joint  performance  of  the  two  tasks  benefited  from  the 
graphical  presentation.  The  issue  of  SA  was  also  addressed  in  a  debriefing  questionnaire.  Subjects  that 
used  graphical  displays  rated  their  ability  to  anticipate  changes  in  automation  level  significantly  higher 
than  subjects  that  used  the  tabular  displays. 

KEYWORDS:  aviation  (military),  avionics,  measurement,  performance-based  measures, 
pilot  aiding,  questionnaire/survey  data,  retrospective  measures,  subjective 
measures,  target  identification 
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13.  Barbato,  G.J.,  and  McCracken,  J.R.  (1993).  Crew  system  design  alternatives  for 
single  seat  fighters.  In  Proceedings  of  the  Seventh  International  Symposium  on 
>1v/af/on  Psyc/jo/og'y  (Volume  1,  pp.  98-102).  Columbus:  The  Ohio  State 
University. 

This  project  is  part  of  an  on-going  program  to  investigate  the  most  effective  approach  for  incorporating 
data-linked  information  into  a  tactical  fighter  cockpit.  The  source  of  the  data-linked  data  might  be 
air-based  or  ground-based  radar  systems,  or  the  sensors  from  other  fighters.  Several  display  options 
were  tested  by  six  active  or  former  military  fighter  pilots.  Both  objective  (i.e.,  deviation  from  flight 
path,  number  of  switch  closures,  etc.)  and  subjective  (questionnaire)  data  were  collected.  The  data 
suggested  that  the  pilots  preferred  the  combination  of  a  Sensor  Volume  Display  (that  showed  the 
portion  of  space  being  scanned  by  the  pilot's  aircraft  and  his  wingman's  aircraft)  and  a  Tactical 
Situation  Format  display  (that  showed  a  symbolic  representation  of  the  battle  area  and  actors  from  a 
bird's-eye  view).  The  subjective  data  also  suggest  that  the  pilots  favor  the  development  of  display 
devices  with  much  larger  display  areas  than  are  currently  available  (i.e.,  big  picture  displays). 

KEYWORDS:  aviation  (military),  avionics,  big  picture  displays,  combat  simulation,  data 
fusion,  data  link,  display  design,  measurement,  performance-based 
measures,  pilot  aiding,  questionnaire/survey  data,  subjective  measures 


14.  Beach,  M,  (1990).  Cooperative  tactical  operations  using  a  data  link.  In 

Proceedings  of  the  lEEE/AIAA/NASA  9th  Digital  Avionics  Systems  Conference  (pp. 
549-551).  New  York:  Institute  of  Electrical  and  Electronics  Engineers. 

The  author  discusses  the  design  requirements  and  benefits  for  a  data  link  system  connecting  combat 
aircraft.  Overall,  he  interprets  the  Unctions  of  a  data  link  system  in  three  categories;  situation 
awareness,  offensive  functions,  and  defensive  functions.  In  introducing  the  SA  benefits  of  data  link, 
the  author  states  (p.  549):  "The  most  fundamental  benefit  of  sharing  information  with  a  wingman  is 
the  increase  in  situation  awareness.  In  order  to  get  this  situation  awareness,  track  files  must  be 
exchanged  as  a  minimum."  Track  files  are  the  information  from  the  aircraft's  sensor  systems  (primarily 
radar)  on  the  attributes  (e  g.,  location,  altitude,  heading,  speed,  identity,  etc.)  of  other  aircraft.  The 
author  points  out  that  for  technological  reasons  this  will  be  the  most  challenging  information  to 
exchange  across  the  data  link,  but  suggests  that  it  is  the  most  essential.  The  impact  on  SA  is  that  the 
pilot  of  an  aircraft  will  gain  information  about  aircraft  that  his  own  sensors  are  not  assigned  to  track  or 
are  unable  to  track. 

KEYWORDS:  air-to-air  mission,  aviation  (military),  avionics,  data  link,  pilot  aiding 
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15.  Beringer,  D.B.,  and  Hancock,  P.A.  (1989),  Exploring  situational  awareness:  A 
review  and  the  effects  of  stress  on  rectilinear  normalization.  In  Proceedings  of  the 
Fifth  internationai  Symposium  on  Aviation  Psychology  (Volume  2,  pp.  646-651). 
Columbus:  The  Ohio  State  University. 

The  authors  start  the  review  with  a  discussion  of  the  lack  of  a  consensus  definition  of  situation 
awareness.  They  then  suggest  that  SA  in  human/machine  systems  be  defined  (p.  646),  "as  conscious 
awareness  of  actions  within  two  mutually  embedded  four-dimensional  envelopes."  The  two  envelopes 
are:  the  "inner"  envelope  (unaided  sensory  space),  and  the  "outer"  envelope  (information  available  via 
remote  sensing).  A  hierarchical  model  of  the  component  processes  required  for  situation  awareness  is 
described.  The  model  has  three  levels:  (1)  A  Higher  Macro-level:  maintaining  orientation,  maintaining 
action  priorities,  integrating  subtasks,  and  monitoring  cues;  (2)  A  Lower  Macro-level:  the  requisite 
skills  and  activities  to  feed  the  necessary  information  to  the  higher  level  (detection,  differentiation, 
diagnosis,  and  inference);  (3)  A  Micro-level:  detection  and  differentiation  of  information  as  influenced 
by  stimulus  factors.  The  authors  suggest  that  there  are  five  potential  methods  for  improving  situation 
awareness  in  the  cockpit:  selection,  training,  system  improvement,  system  design,  and  compensation 
for  stress.  As  an  example  of  a  componential  approach  to  situation  awareness  research,  the  authors 
describe  a  laboratory  study  that  was  designed  to  examine  the  relationship  between  stress  (inflicted  by 
varying  time-sharing  demands)  and  spatial  memory  of  a  course.  Only  preliminary  results  were 
presented,  but  based  on  these  data  the  authors  conclude  (p.  650)  "It  is  clear  that  training  or  experience 
is  useful  in  facilitating  performance  of  this  type  of  task,  suggesting  that  enhancement  of  component 
skills  through  training  may  be  one  route  to  the  overall  enhancement  of  SA." 

KEYWORDS:  cognitive  processes,  definition,  measurement,  retrospective  measures, 
subjective  measures,  training 


16.  Betts,  F.C.  (1987,  January).  Situational  Awareness  Study  (SAS),  Volume  1, 
Executive  summary  (Tech.  Report  No.  AFWAL  TR-86-1 150-Vol-1). 
Wright-Patterson  Air  Force  Base,  OH:  Air  Force  Wright  Aeronautical  Laboratories. 
(AD-B121972) 

The  main  product  of  this  study  was  software  which  facilitates  data  reduction  and  evaluation  of 
man-in-the-loop  simulations  of  fire  control  systems.  The  study  evaluates  data  from  an  advanced 
medium-range  air-to-air  missile  operational  utility  evaluation  (AMRAAM  OUE)  and  from  a 
multi-sensor  integration  (MSI)  system  simulation.  The  SAS  software  helped  illustrate  system 
requirements  which  will  lead  to  two  specific  improvements  in  pilot  situational  awareness  of 
adversaries:  (1)  better  weapons  usage,  and  (2)  higher  survivability  rates, 

KEYWORDS:  air-to-air  mission,  aviation  (military),  avionics,  combat  simulation,  data 
fusion,  measurement,  performance-based  measures 
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17.  Biferno,  M.A.,  and  Stanley,  D.L.  (1983).  The  touch-sensitive  control/dispiay  unit: 
A  promising  computer  interface  (SAE  Paper  No.  831532).  Warrendale,  PA: 
Society  of  Automotive  Engineers. 

The  authors  present  a  description  of  a  new  interface  device  called  a  touch-sensitive  control/display  unit 
(touch-CDU).  The  touch-CDU  allows  the  system  operator  or  pilot  to  interact  with  a  system  by 
touching  the  display  elements  which  could  then  be  changed  to  present  new  information  or  to  offer  new 
options.  Although  neither  the  data  collection  nor  analysis  procedures  are  described  within  this  paper, 
the  authors  state  (p.  41 1):  "Preliminary  work  has  indicated  that  a  great  improvement  in  crew  workload 
and  increased  situational  awareness  can  be  obtained  by  the  thoughtful  application  of  touch- 
CDUs  to  certain  tasks." 

KEYWORDS:  aviation  (civilian),  display  design,  workload 


18.  Blanchard,  R.E.  (Panel  Chair)  (1993).  Situation  Awareness:  Transition  from  theory 
to  practice.  In  Proceedings  of  the  Human  Factors  and  Ergonomics  Society  37th 
Annual  Meeting  (Volume  1,  pp.  39-42).  Santa  Monica,  CA:  Human  Factors  and 
Ergonomics  Society. 

This  set  of  abstracts  summarizes  a  panel  session.  A  unifying  theme  to  the  session  was  that  the 
theoretical  framework  for  SA,  while  not  complete,  was  sufficient  to  support  research  in  applying  the 
SA  concept.  Four  abstracts  follow  the  introduction:  Mica  Endsley  {Situation  Awareness:  The 
Development  and  Application  of  a  Theoretical  Framework),  Richard  Mogford  {The  Importance  of 
Situation  Awareness),  Mark  Smolensky  {Toward  the  Physiological  Measurement  of  Situation 
Awareness:  The  Case  for  Eye  Movement  Measurements),  and  Mark  Rodgers  {Use  of  Incident 
Re-creation  for  Studying  Situation  Awareness). 

KEYWORDS:  attention,  cognitive  processes,  decision  making,  expertise,  measurement, 
memory  probe  measures,  mental  models,  performance-based  measures, 
physiological  measures 
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19.  Bolstad,  C.A.  (1991).  Individual  pilot  differences  related  to  situation  awareness.  In 
Proceedings  of  the  Human  Factors  Society  35th  Annual  Meeting  (Volume  1,  pp. 
52-56).  Santa  Monica,  CA:  Human  Factors  Society. 

The  author  cites  previous  research  that  had  demonstrated  that  large  individual  differences  existed  in  the 
generation  and  maintenance  of  SA.  This  implied  that  SA  "may  be  trainable"  (p.  52).  The  research  in 
this  paper  involved  giving  21  experienced  military  pilots  a  battery  of  cognitive  tests  to  determine  if  any 
specific  abilities  correlated  with  SA  measured  in  a  simulator.  Based  on  a  literature  review  conducted  by 
the  author,  the  tests  in  the  battery  were  selected  to  assess  the  subjects'  abilities  in  six  domains:  spatial 
abilities,  attention,  memory,  perceptual  abilities,  cognition,  and  personality.  SA  was  assessed  by  an 
abbreviated  version  of  SAGAT.  Some  significant  correlations  between  performance  on  battery  tests 
and  SA  in  the  simulation  were  discovered.  The  largest  correlation  was  between  the  test  of  attention 
sharing  ability  and  SA,  followed  by  tests  of  immediate/delayed  memory  and  encoding  speed.  The 
author  proposes  further  work  to  explore  these  relationships  and  suggests  (p.  55)  that  "Future 
evaluations  will  lead  to  the  development  of  instructional  methods  for  enhancing  SA  and  could  help 
identify  the  most  promising  areas  for  research  and  development  in  SA  training." 

KEYWORDS:  attention,  aviation  (military),  cognitive  processes,  combat  simulation, 
individual  differences,  measurement,  memory  probe  measures, 
performance-based  measures,  SAGAT,  training 


20.  Bond,  D.F.  (1992,  May  11).  Army  evaluating  Longbow,  weighs  broader  program. 
Aviation  Week  &  Space  Technology,  /35(19),  42-44. 

The  article  describes  proposed  changes  to  the  Army's  AH-64A  attack  helicopters  to  upgrade  them  to 
AH-64C/D  Apache  Longbow  helicopters.  The  proposed  changes  include  extensive  upgrading  of  the 
avionics  with  a  new  millimeter-wave  radar,  communications  and  navigational  improvements,  and  an 
improved  data  modem  (IDM).  The  IDM  is  considered  to  be  a  key  interface  for  the  Longbow  system. 
The  potential  for  this  system  was  described  by  Charles  Vehlow  (McDonnell  Douglas  chief  of  AH-64 
derivatives  programs)  as  follows  (p.  44),  "The  AH-64A's  target  acquisition  and  designation  system 
limits  situational  awareness  to  consideration  of  individual  targets  in  sequence."  By  contrast,  the 
Longbow  system  will  detect  well  over  a  dozen  targets,  classify  them  by  type  (armor,  wheeled  vehicles, 
air  defense  weapons  and  the  like)  and  prioritize  them  according  to  how  threatening  they  are.  All  this 
information  will  be  shown  to  crewmembers  on  the  tactical  situation  display."  By  using  the  EDM  to 
exchange  this  information  with  other  helicopters,  the  Longbow  helicopter  can  act  as  a  "battle  captain" 
to  ensure  effective  use  of  the  available  forces  and  avoid  attacks  on  friendly  forces. 

KEYWORDS:  avionics,  data  fusion,  display  design,  fratricide,  helicopter,  pilot  aiding, 
target  identification 
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2i.  Bowers,  C.A,,  Braun,  C.,  and  Kline,  P.B.  (1994).  Communication  and  team 
situational  awareness.  In  R.C.  Gilson,  D.J.  Garland,  and  J.M.  Koonce  (Eds.), 
Situational  awareness  in  complex  systems:  Proceedings  of  a  CAHFA  Conference 
(pp,  305-311).  Daytona  Beach,  FL:  Embry-Riddle  Aeronautical  University  Press. 

This  study  investigated  the  relationship  between  communications  patterns  and  performance  of  teams  in 
simulated  flight  scenarios  that  were  designed  to  demand  high  levels  of  SA.  The  subjects  were 
IFR-rated  pilots.  The  High-SA  flight  was  simulated  using  Flight  Simulator  4.0  on  a  desktop  computer 
system.  The  High-SA  flight  was  a  low  level  reconnaissance  flight  that  required  subjects  to  identify 
buildings.  The  various  teams  were  divided  into  higii  and  low  performance  groups.  The 
communications  patterns  of  these  groups  were  contrasted.  No  statistically  significant  differences  were 
detected  between  the  two  groups  in  communications  rate  in  any  of  the  predefined  categories. 

However,  there  was  some  evidence  that  the  pattern  of  occurrence  of  statements  from  the  different 
communications  categories  could  discriminate  between  the  low  and  high  performance  groups.  The 
authors  concluded  (p.  310):  ‘  These  results  Indicate  that  pattern  analysis  of  communications  is  a 
promising  tool  for  understanding  the  nature  of  SA  in  teams.  However,  it  should  be  noted  that  these 
results  are  based  on  a  very  small  sample  si7.e  flying  one  relatively  unique  type  of  mission.  Future 
research  should  assess  the  degree  to  which  these  results  generalize  across  aircraft  and  mission 
requirements." 

KEYWORDS,  aviation  (civilian),  measurement,  subjective  measures,  team  SA,  verbal 
protocols 


22.  Braune,  R.  (1987).  Surrimary  of  the  workshop  on  cockpit  automation  in  commercial 
airplanes.  In  Proceedings  of  the  Fourth  International  Symposium  on  Aviation 
Psychology  (pp.  9-15).  Columbus:  The  Ohio  State  University, 

Tliis  workshop  convened  to  consider  persisting  problems  in  the  application  of  automation  to 
commercial  airplane  cockpits.  The  first  issue  listed  for  them  to  consider  (on  a  list  of  10  issues)  was 
"Loss  of  situational  awareness."  In  the  summary  of  the  discussions  by  the  panel,  the  issue  of  SA  was 
presented  in  the  context  of  display  technology  and  the  pilot's  understanding  of  what  the  automation  is 
doing  (p.  13):  "Every  effort  should  be  made  to  use  display  technologies  to  help  the  pilot  develop  and 
maintain  a  maximum  of  situational  awareness.  A  basic  assumption  is  that  the  pilot  should  be  able  to  fly 
the  airplane  manually  or  monitor  automatic  flight  with  an  equal  level  of  efficiency.  Only  if  this  can  be 
accomplished  can  we  be  assured  that  the  pilot's  monitoring  of  the  automatic  systems  is  effective,  i.e., 
the  pilot  fiilly  understands  what  the  automatic  system  is  doing." 

KEYWORDS:  aviation  (civilian),  avionics,  display  design,  pilot  aiding 
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23.  Brookes,  A.  (1991) •  Crash!  Military  aircraft  disasters,  accidents  and  incider)ts. 
London:  Ian  Allan  Ltd. 

This  book  is  not  explicitly  about  SA,  but  SA  is  mentioned  occasionally.  The  book  reviews  numerous 
military  aircraft  accidents,  many  of  which  involve  lack  of  SA.  Of  particular  interest  is  the  chapter 
"Finger  Trouble"  which  discusses  an  attempt  by  the  RAF  to  increase  "flight  safety  awareness"  during 
World  War  II.  The  effort  involved  the  creation  of  a  fictional  cartoon  character,  Pilot  Officer  Prune, 
who  illustrated  many  unsafe  activities  in  a  humorous  way.  Among  other  re.sponsibilities.  Pilot  Officer 
Prune  was  Patron  of  the  Most  Highly  Derogatory  Order  of  the  Irremovable  Finger,  a  monthly  award 
commemorating  outstanding  examples  of  doing  the  "wrong  thing  at  the  wrong  time  wliile  flying"  (p. 
45).  This  led  to  the  use  of  the  term  "finger  trouble"  as  an  admonition  to  a  pilot  or  crew  member  to  get 
his  act  together.  No  data  are  presented  to  demonstrate  that  this  effort  increased  flight  safety  in  the 
RAF,  but  "finger  trouble"  seems  like  a  precursor  to  the  concept  of  SA. 

KEYWORDS:  aviation  (military),  pilot  error 


24.  Bunecke,  J.L.,  Povenmire,  H.K.,  Rockway,  M.R.,  and  Patton,  M.W.  (1990,  April). 
The  situational  awareness  component  of  cockpit  resource  management.  In 
Proceedings  of  the  Psychology  in  the  Department  of  Defense  Twelfth  Symposium 
(Tech.  Report  No.  USAFA-TR-90-1 ,  pp.  259-263).  Colorado  Springs,  CO:  US  Air 
Force  Academy.  (AD-A2401 1 3) 

This  article  investigated  the  role  of  crew  communication  in  maintaining  SA  in  a  simulated  B-52  combat 
mission.  During  the  simulated  mission,  all  communications  among  the  B-52  crew-members  were 
recorded  and  then  analyzed  for  their  contribution  to  Cockpit  Resource  Management  (CRM). 

The  frequency  of  the  various  classes  of  messages  was  correlated  with  the  ratings  of  crew  performance 
as  assessed  by  expert  observers  and  bombing  accuracy.  Overall,  the  amount  of  crew  communication 
was  found  to  correlate  positively  with  the  expert  ratings  of  crew  performance.  The  authors  makes 
suggestion  about  how  communication  can  be  effectively  used  to  increase  crew  SA  and  distribute  crew 
workload. 

KEYWORDS:  air-to-ground  mission,  aviation  (military),  cockpit  resource  management 
(CRM),  combat  simulation,  measurement,  performance-based  measures, 
subjective  measures,  team  SA,  verbal  protocols,  workload 
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25.  Busquets,  A.M.,  Parrish,  R.V.,  Williams,  S.P.,  and  Nold,  D.E.  (1994).  Comparison 
of  pilots'  acceptance  and  spatial  awareness  when  using  EFIS  vs.  pictorial  display 
formats  for  complex,  curved  landing  approaches.  In  R.D.  Gilson,  D.J.  Garland, 
and  J.M.  Koonce  (Eds.),  Situational  awareness  in  complex  systems:  Proceedings 
of  a  CAHFA  Conference  (pp.  139-167).  Daytona  Beach,  FL:  Embry-Riddie 
Aeronautical  University  Press. 

This  paper  reviews  an  investigation  of  the  effects  of  using  pictorial  "pathway  in  the  sky"  displays 
rather  than  the  current  Electronic  Flight  Infonnation  Systems  (EFIS)  displays  (with  and  without  a 
Flight  Director  on  the  spatial  awareness  component  of  situation  awareness.  The  experiment  was 
conducted  in  a  simulator  using  16  commercial  airline  pilots  as  subjects.  Situation  awareness  was 
measured  by  a  variety  of  means:  aircraft  control  performance,  anomalous  cues/detection  time  (a 
variation  of  an  implicit  technique),  two  versions  of  probe  techniques  (including  a  memory  probe 
measure),  and  subjective  questionnaires  (including  questions  concerning  workload).  Overall,  the 
results  supported  the  value  of  the  pictorial  "pathway  in  the  sky"  displays.  These  displays  appeared  to 
increase  spatial  awareness  while  reducing  the  workload  experienced. 

KEYWORDS:  aviation  (civilian),  display  design,  implicit  measures,  measurement,  memory 
probe  measures,  performance-based  measures,  questionnaire/survey  data, 
subjective  measures,  workload 


26.  Calhoun,  G.L.,  Janson,  W.P.,  and  Valencia,  G.  (1988).  Effectiveness  of 

three-dimensional  auditory  direcrional  cues.  In  Proceedings  of  the  Human  Factors 
Society  32nd  Annual  Meeting  (Volume  1,  pp.  68-72).  Santa  Monica,  CA:  Human 
Factors  Society, 

This  research  investigated  the  potential  benefits  of  3-D  audio  in  the  cockpit.  More  specifically,  the 
authors  suggested  that  3-D  audio  could  provide  three  advantages:  (1)  increasing  situational  awareness 
of  attitude,  threats,  and  terrain,  (2)  enhanced  communication  intelligibility,  and  (3)  cuing  on  where  to 
look  for  peripheral  cues.  The  experiment  investigated  the  cuing  properties  of  3-D  audio.  The  results 
supported  the  utility  of  3-D  audio.  Localized  auditory  cues  promoted  improved  acquisition  of 
peripheral  cues  compared  to  either  coded  tones  or  speech  messages.  The  authors  concluded  that  (p. 
72),  "The  results  from  the  present  study  suggest  that  the  use  of  3-D  audio  aural  signals  as  natural 
dii  ectional  cues  will  allow  the  pilot  to  quickly  and  naturally  construct  a  mental  image  of  the  situation 
with  little  expenditure  of  mental  processing  resources." 

KEYWORDS:  3'D  audio  displays,  attitude  displays,  display  design,  measurement, 
performance-based  measures 
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27.  Canan,  J.W.  (1993,  April).  Space  support  for  the  shooting  wars.  Air  Force 
Magazine,  76(A),  30-34. 

General  Charles  A.  Homer  was  the  three-star  air  component  commander  of  U.S.  Central  Command 
(CENTCOM)  and  of  allied  coalition  air  forces  during  the  Gulf  War,  This  article  reviews  his  approach 
to  his  new  job  as  the  four-star  General  commander-in-chief  of  the  multi-service  U.S.  Space  Command. 
A  major  theme  throughout  General  Homer's  plans  for  Space  Command  is  improving  the  direct 
communication  between  satellite  intelligence  assets  and  the  soldier  or  pilot  at  the  front  of  a  battle. 
Among  the  systems  mentioned  are  Global  Positioning  System  (GPS)  terminals  for  the  cockpits  of 
combat  aircraft  and  the  Talon  Sword  data  link.  The  benefits  of  such  systems  are  justified  by  the 
expected  increase  in  aircrew  situation  awareness.  General  Homer  expects  that  the  increase  in  situation 
awareness  that  results  from  such  systems  will  increase  combat  effectiveness  and  reduce  the  probability 
of  fratricide. 

KEYWORDS:  aviation  (military),  avionics,  data  link,  fratricide,  pilot  aiding 


28.  Carmody,  M.A.,  and  Gluckman,  J.P.  (1993).  Task  specific  effects  of  automation 
and  automation  failure  on  performance,  workload,  and  situational  awareness.  In 
Proceedings  of  the  Seventh  internationai  Symposium  on  Aviation  Psychoiogy 
(Volume  1,  pp.  167-171).  Columbus:  The  Ohio  State  University. 

The  research  investigated  the  effect  of  the  nature  of  an  operator's  mental  model  on  recovery  from  a 
simulated  automation  failure.  Tasks  were  divided  into  two  categories:  stable  and  dynamic.  Stable 
ta.sks  were  those  in  which  the  information  relevant  for  decision  makijig,  and  especially  diagnosis,  did 
not  change  over  time.  Dynamic  tasks  were  those  in  which  the  information  required  for  decision  making 
does  change  over  time.  It  was  hypothesized  that  the  SA  loss  produced  by  an  automated  system  would 
be  more  profound  for  dynamic  tasks.  Two  laboratory  studies  using  the  Multiattribute  Task  (MAT) 
battery  were  conducted  to  test  the  hypothesis.  In  one  of  the  studies,  the  Situation  Awareness  Global 
Assessment  Technique  (SAGAT)  was  used  as  a  metric  of  situation  awareness.  Based  on  the  results, 
the  authors  concluded  that  the  stability  of  the  internal  cognitive  model  was  an  important  distinction  in 
interpreting  the  effects  of  automation. 

KEYWORDS:  cognitive  processes,  decision  aiding,  decision  making,  measurement, 

memory  probe  measures,  mental  models,  performance-based  measures, 
SAGAT 
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29.  Carr,  L.A.,  and  Lopina,  R.F.  (1982).  Pilot  workload  in  the  night  attack  mission.  In 
Proceedings  of  the  IEEE  1982  National  Aerospace  and  Electronics  Conference  - 
NAECON  1982  (Volume  2,  881-884).  New  York:  Institute  of  Electrical  and 
Electronics  Engineers. 

The  paper  reviews  the  work  conducted  by  the  Night  Attack  Workload  Steering  Group  (NAWSG).  The 
NAWSG  was  concerned  with  the  implementation  of  night-vision  de^'^ces  (such  as  the  Low  Altitude 
Navigation  and  Targeting  for  Night  (LANTIRN)  system)  to  fighter  and  attack  aircraft.  A  major 
problem  with  implementing  these  systems  was  perceived  to  be  the  extremely  high  workload  incurred 
by  their  use.  The  workload  in  such  aircraft  was  seen  to  be  closely  related  to  pilot  SA.  For  example,  in 
discussing  possible  solutions  the  authors  state  (p.  883);  "One  of  the  key  and  potentially  the  most 
significant  factors  in  reducing  pilot  workload  is  to  enhance  the  pilot's  situational  awareness.  Some  of 
the  more  lucrative  systems  for  consideration  are  moving  map  displays,  integrated  color  displays,  wide 
field  of  view  heads  up  displays,  and  a  very  accurate  inertial  navigation  system." 

KEYWORDS:  aviation  (military)  avionics,  display  design,  workload 


30.  Carretta,  T.R.,  Perry,  D.C.,  and  Ree,  M.J.  (1994).  The  ubiquitous  three  in  the 
prediction  of  situational  awareness:  Round  up  the  usual  suspects.  In  R.D.  Gilson, 
D.J.  Garland,  and  J.M.  Koonce  (Eds.),  Situational  awareness  in  complex  systems: 
Proceedings  of  a  CAHFA  Conference  (pp.  125-137).  Daytona  Beach,  FL: 
Embry-Riddle  Aeronautical  University  Press. 

This  paper  reviews  the  design  of  an  investigation  of  developing  SA  selection  tools  for  use  by  the  US. 

Air  Force.  Previous  attempts  to  study  the  use  of  selection  tests  for  SA  abilities  have  suffered  from 
limited  sample  size.  This  research  will  use  a  sample  of  about  200  F-15  pilots  to  provide  a  more 
reasonable  level  of  statistical  power.  A  large  battery  of  selection  tasks  v'ill  be  tested.  The  tasks  will  be 
based  on  a  fundamental  set  of  the  "ubiquitous  three"  in  selection  research  (i.e.,  psychometric  g, 
psychomotor  skill,  and  the  personality  construct  of  "conscientiousness")  and  supplemented  by  various 
tasks  suggested  in  the  SA  literature.  The  task  battery  will  combine  both  performance-based  tests  and 
subjective  measures  as  potential  correlates  of  SA  abilities.  The  data  from  these  tasks  will  be  tested 
against  a  criterion  SA-rating  collected  from  the  pilots,  their  commanders,  and  their  peers.  At  the  time 
of  the  writing,  the  data  collection  was  not  complete,  so  no  data  are  presented.  (NOTE  -  this  project  is 
part  of  the  SAINT  team  research  reviewed  by  McMillan  in  this  Technical  Report.) 

KEYWORDS:  aviation  (military),  cognitive  processes,  individual  differences,  measurement, 
performance-based  measures,  subjective  measures 
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31  Carroll,  L.A.  (1992,  March).  Desperately  seeking  SA.  TAG  Attack  (TAC  SP 
127-1),  32(3),  5-6. 

The  author  discusses  the  importance  of  situation  awareness  to  Air  Force  operations.  An  Air  Staff 
process  action  team  was  formed  to  answer  the  questions  (p,  5);  "Just  what  do  we  mean  by  situational 
awareness?  Can  it  be  measured  objectively?  Can  SA  be  learned?  Can  we  select  for  it?  If  it  can  be 
measured,  when  in  the  flying  training  process  should  we  take  measurements?"  The  Air  Staff  team 
proposed  the  following  operationally-  oriented  definition  for  situation  awareness  (p.  6),  "a  pilot's  (or 
aircrew's)  continuous  perception  of  self  and  aircraft  in  relation  to  the  dynamic  environment  of  flight, 
threats,  and  mission,  and  the  ability  to  forecast,  then  execute  tasks  based  on  that  perception."  The 
author  points  out  that  current  programs  involving  the  training  of  spatial  orientation,  task  and  attention 
management,  crew  coordination,  and  so  on,  are  related  to  the  creation  and  maintenance  of  .situation 
awareness.  However,  these  are  not  considered  to  be  sufficient  and  the  author  proposes  that  (p.  5),  "SA 
should  be  the  umbrella  under  which  applicable  human  factors  research  and  training  are  pursued." 

KEYWORDS:  attention,  aviation  (military),  cognitive  processes,  definition,  training 


32.  Chandra,  D.,  and  Bussolari,  S.R.  (1991).  An  evaluation  of  flight  path  management 
automation  in  transport  category  aircraft.  In  Proceedings  of  the  Sixth 
internationai  Symposium  on  Aviation  Psychoiogy  (Volume  1,  pp.  139-144). 
Columbus:  The  Ohio  State  University. 

Desktop  simulations  of  a  Boeing  1511161  Electronic  Flight  Instrumentation  System  (EFIS)  and  Control 
Display  Unit  (CDU)  were  used  to  evaluate  the  effect  of  varying  the  communication  method  of 
providing  clearance  amendments.  Amendments  were  presented  either  via  the  traditional  speech 
commands,  textual  presentation  on  the  display,  or  graphical  presentation  on  the  display.  The  graphical 
display  was  expected  to  be  superior  because  it  was  anticipated  that  the  graphical  display  symbols 
would  be  more  similar  to  the  pilots'  internal  representations  of  the  situation.  The  pilots  willingness  to 
accept  unacceptable  clearance  amendments  was  used  as  an  indication  of  poor  SA.  Although  the  results 
failed  to  meet  statistical  significance,  the  graphical  display  tended  to  reduce  the  time  needed  for 
comprehension  and  reduce  subjective  workload  ratings.  The  graphical  mode  was  also  rated  the  highest 
on  a  preference  scale.  However,  there  was  no  indication  that  the  mode  of  presentation  affected  the 
detection  of  unacceptable  clearance  amendments. 

KEYWORDS:  3"'  traffic  control,  aviation  (civilian),  avionics,  cognitive  processes,  display 
design,  measurement,  mental  models,  performance-based  measures,  pilot 
aiding,  subjective  measures,  workload 
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33.  Companion,  M.A.  (1994).  Situational  awareness  in  emergency  management 
systems:  An  overview.  In  R.D.  Gilson,  D.J.  Garland,  and  J.M.  Koonce  (Eds.), 
Situational  awareness  in  complex  systems:  Proceedings  of  a  CAHFA  Conference 
(pp.  283-289).  Daytona  Beach,  FL:  Embry-Riddle  Aeronautical  University  Press. 

The  author  describes  SA  as  (p.  284);  "the  process  by  which  a  person  extracts,  integrates,  assesses  and 
responds  to  task  relevant  information  from  the  total  environment,  both  spatial  and  temporal."  The 
author  suggests  and  explains  that  S  A  and  complex  decision  making  skills  might  be  usehjl  concepts  to 
apply  to  the  training  of  people  involved  in  emergency  management  (e.g.,  a  fire  chief,  hazardous  material 
control,  etc.).  To  accomplish  this,  it  is  proposed  that  the  simulation  capabilities  of  desktop  computers 
be  used  as  training  aids  to  familiarize  people  with  the  process  of  responding  to  emergencies  in  their 
domain. 

KEYWORDS:  cognitive  processes,  decision  making,  definition,  expertise,  training 


34.  Cooper,  G.  (1991,  November).  Tomorrow's  cockpit  displays.  Aerospace,  /5(11), 
12-14. 

The  article  reviews  a  lecture  by  R.A.F.  Fit  Lts  A.C.B.  Singer  and  I.W.  McClelland  that  reviewed  aircraft 
display  research  conducted  at  Royal  Aerospace  Establishment  Bedford.  One  project  that  is  discussed  is 
the  development  of  the  Bedford  Fast  Jet  Format  (FJF)  for  head-up  displays  (HUDs).  The  FJF  was 
adopted  as  a  standard  for  the  RAF  in  1988  and  was  selected  for  the  European  Fighter  Aircraft  (EFA) 
program.  In  discussing  the  outcome  of  the  development  program,  it  was  stated  that,  "The  pitch  ladder 
assists  orientation  and  the  FJF  as  a  whole  promotes  increased  mission  effectiveness  by  increasing  the 
pilot's  situation  awareness  and  thus  reducing  his  workload"  (p.  13).  In  the  description  of  future 
programs,  the  maintenance  or  improvement  of  SA  is  presented  as  a  primary  goal. 

KEYWORDS:  attitude  displays,  aviation  (military),  display  design,  pilot  aiding,  workload 


35.  Corwin,  W.H.,  Probert,  A.,  and  Royer,  R.  (1993).  Synthetic  terrain  imagery  for 
helmet-mounted  display.  In  Proceedings  of  the  Seventh  International  Symposium 
on  Aviation  Psychology  (Volume  1,  pp.  108-114).  Columbus:  The  Ohio  State 
University. 

The  article  reports  the  results  of  a  study  on  the  effects  of  displaying  a  synthetic  representation  of 
terrain  in  the  background  of  a  helmet-mounted  display  on  the  spatial  situation  awareness  of  pilots.  The 
goal  was  not  to  create  a  virtual  world,  but  to  determine  the  minimum  necessary  visual  cues.  Several 
possible  formats  for  presenting  the  synthetic  terrain  were  compared.  The  dependent  data  were 
subjective  assessments  from  ten  pilots  who  observed  the  different  displays  in  a  F- 15  simulator.  The 
preferred  format  was  called  the  "Mesh"  display.  The  Mesh  display  (p.  109),  "looks  as  though  a  large 
net  was  laid  across  the  earth." 

KEYWORDS:  aviation  (military),  combat  simulation,  display  design,  helmet-mounted 
displays  (HMDs),  measurement,  pilot  aiding,  subjective  measures 


67 


36.  Covault,  C.  (1992,  September  14).  Russian  aircraft  marketing  threatens  power 
balance.  Avietion  Week  &  Space  Technology,  137(^^),  22-23. 

In  a  discussion  of  Russian  attempts  to  market  advanced  military  aircraft,  this  article  discusses  changes 
in  the  MiG-29  designed  to  increase  its  combat  effectiveness  and  salability.  The  MiG-29M  has  already 
added  two  multifunction  computer  displays  (MFDs)  to  the  MiG-  29  cockpit.  In  discussing  future  plans 
for  the  aircraft,  the  article  notes  (p.  23),  "the  Russians  are  examining  more  complex  tactical  situation 
displays,  including  color  callouts,  to  project  on  the  MFDs.  These  would  provide  extremely  useful 
situational  awareness  data  to  the  pilot  preparing  for  an  engagement." 

KEYWORDS,  aviation  (military),  display  design,  pilot  aiding 


37,  Crabtree,  M.S.,  Marcelo,  R.A.Q.,  McCoy,  A.L.,  and  Vidulich,  M.A.  (1993).  An 
examination  of  a  subjective  situational  awareness  measure  during  training  on  a 
tactical  operations  simulator.  In  Proceedings  of  the  Seventh  International 
Symposium  on  Aviation  Psychology,  (Volume  2,  pp.  891-895).  Columbus:  The 
Ohio  State  University. 

This  paper  described  an  examination  of  the  situation  awareness  rating  technique  (SART)  for  sensitivity 
to  variables  which  may  affect  SA.  The  10-dimensional  version  of  SART  was  used,  with  an  added 
eleventh  dimension  for  overall  SA.  Variables  studied  were  out-the-window  display  type  (4  "daylight" 
displays  characterized  by  color  of  tank,  forward  looking  infrared  (FLIR),  and  Global  Positioning 
Systems  (GPS)  displays),  presence/absence  of  surface-to-air  missile  (SAM)  sites,  approach  scenario, 
and  size  of  target  (tarJe)  vulnerable  area.  Six  male  subjects  were  trained  for  20  hours  on  the  Simulator 
for  Tactical  Operations  and  Research  Measurement  (STORM)  system;  SART  measures  were  solicited 
twice  for  each  condition,  early  and  late  in  the  training  period.  SART  dimensions  showed  sensitivity  to 
all  variables  except  size  of  target's  vulnerable  area.  SART's  demand  on  attentional  resources  subscales 
(instability,  variability,  and  complexity)  showed  sensitivity  to  more  variables  than  other  subscales. 
Presence/absence  of  SAMs  had  the  largest  effect  on  these  demand  subscales.  Many  of  the  SART 
dimensions  were  sensitive  to  display  type,  indicating  higher  SA  when  red  or  black  tank  displays  were 
used.  In  addition,  SART  was  sensitive  to  whether  the  rating  was  given  early  or  late  in  the  training 
period;  as  expected,  subjects  indicated  far  fewer  demands  on  attentional  resources  when  operating  the 
simulator  later  in  the  training  period. 

KEYWORDS:  air-to-ground  mission,  attention,  aviation  (military),  cognitive  processes, 
combat  simulation,  measurement,  SART,  subjective  measures,  training 
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38.  Craig,  J.L.,  and  Purvis,  B.D.  (1990).  6*52  Night  vision  goggle  head-up  display 
development.  In  Proceedings  -  Cockpit  Dispioys  and  Visual  Simulation  (SPIE 
Proceedings  Series  Volume  1289,  pp  63-71).  Bellingham,  WA:  Society  of 
Photo-Optical  Instrumentation  Engineers. 

This  paper  describes  the  implementation  of  a  new  night  vision  display  into  the  US  Air  Force  B-S2 
Bomber.  The  display  system  combined  night-vision  goggles  and  a  head-up  display.  As  described  by 
the  authors  (p.  63);  "This  system  displays  flight  and  navigation  information  onto  a  combiner  glass 
which  is  mounted  to  one  of  the  NVG  objective  lenses.  This  allows  the  pilot  to  have  an  'eyes  out' 
orientation,  thereby  decreasing  communication  and  workload,  and  increasing  mission  safety,  situational 
awareness,  and  mission  effectiveness."  The  conclusions  about  the  effects  of  the  system  are  apparently 
based  on  the  comments  of  test  pilots  that  used  the  system  during  actual  night-flight  low-level  missions 
in  a  B-S2.  The  data  collection  procedure  or  results  are  not  discussed  in  enough  detail  to  identify  the 
specific  type  of  tool  used  in  the  evaluation. 

KEYWORDS:  air-to-ground  mission,  aviation  (military),  display  design,  flight  test,  pilot 
aiding,  subjective  measures,  workload 


39.  Crane,  P.M.  (1992,  April).  Theories  of  expertise  as  modeis  for  understanding 
situation  awareness.  In  Proceedings  of  the  Psychology  in  the  Department  of 
Defense  Thirteenth  Symposium  (Tech.  Report  No.  USAFA-TR-92-2,  pp.  148-152). 
Colorado  Springs,  CO:  US  Air  Force  Academy.  (AD-A253006) 

The  author  expresses  concern  that  situation  awareness  as  a  term  that  came  out  of  the  operational 
community  lacks  clear  definition  or  a  clear  connection  to  cognitive  theory.  No  formal  definition  is 
offered,  but  the  author  suggests  that  the  concept  of  expertise  may  provide  a  good  framework  for 
bringing  situation  awareness  into  cognitive  psychology.  Further,  the  possibility  that  skilled  memory 
theory  could  provide  the  model  for  the  cognitive  processing  of  situation  awareness  in  experts  is 
suggested.  If  expertise  and  skilled  memory  are  the  correct  theoretical  background  for  situation 
awareness,  the  author  argues  that  there  are  three  important  implications:  (1)  Situation  awareness 
cannot  be  considered  to  be  an  independent  skill  or  mental  ability.  Therefore,  attempts  to  directly  train 
situation  awareness  will  be  unprofitable.  (2)  Situation  awareness  will  be  domain  specific.  (3) 
Simulator-based  training  for  improving  situation  awareness  should  emphasize  instructional  support  for 
encoding  and  interpreting  simulated  combat. 

KEYWORDS:  cognitive  processes,  definition,  expertise,  training 
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40.  Curran,  J.  (1992).  Trends  in  advanced  avionics.  Ames:  Iowa  State  University 
Press. 

The  book  describes  and  reviews  many  of  the  major  trends  in  modem  avionics  in  both  commercial  and 
military  settings.  Topics  include  the  trend  towards  greater  subsystem  integration,  human-centered 
automation,  artificial  intelligence  aids,  new  display  formats,  navigational  aids,  data-link 
communications,  etc.  In  many  cases,  one  of  the  suggested  advantages  for  the  modem  technology  is 
enhanced  SA. 

KEYWORDS:  aviation  (civilian),  aviation  (military),  avionics,  data  link,  display  design, 
pilot  aiding 


41.  Dalrymple,  M.A.  (1991,  November).  Evaluating  airborne  warning  and  control 
system  strategy  and  tactics  as  they  relate  to  simulated  mission  events  (Tech. 
Report  No.  AL-TP- 199 1-0049).  Brooks  Air  Force  Base,  TX:  Armstrong  Laboratory. 
(AD-A242820) 

The  article  discusses  the  decision-making  processes  of  a  Weapons  Director  (WD)  in  an  AW  ACS 
aircraft.  The  WD  is  responsible  for  directing  the  tactical  aircraft  in  the  region  as  they  execute  their 
missions.  The  first  component  of  the  strategy  used  by  effective  WDs  was  described  as  "developing 
Situational  Awareness"  (p.  i).  The  role  SA  plays  was  described  as  follows  (p.  7),  "To  effectively  deal 
with  events  in  an  air  defense  scenario,  WDs  must  maintain  an  accurate  picture  of  the  battle.  This 
picture  defines  WD  awareness  of  the  current  situation.  Through  situational  awareness,  WDs  choose 
among  the  tasks  competing  for  their  attention  and  then  execute  the  most  important.  This 
decision-making  process  is  more  than  the  application  of  a  predetermined  set  of  priorities....  The  choice 
is  a  trade-off  among  what  will  increase  options  in  the  future,  what  will  ease  future  workload,  and  what 
will  be  the  easiest  to  implement."  This  description  of  the  decision  making  process  of  the  WDs  appears 
to  be  consistent  with  the  notions  of  recognition-primed  decision  making  (refer  to  Klein,  1989a,  1989b). 

KEYWORDS:  aviation  (military),  cognitive  processes,  decision  making 


42.  Deakin,  R.S.  (1990).  Military  aircrew  head  support  system.  In  Proceedings  of  the 
17th  Congress  of  the  International  Council  of  the  Aeronautical  Sciences  -  ICAS 
1990  (pp.  162-167).  Washington,  DC:  American  Institute  of  Aeronautics  and 
Astronautics. 

The  article  discusses  the  Military  Aircrew  Head  Support  System  (MAHSS)  being  developed  in  Britain. 
MAHSS  is  developing  a  head  support  that  could  be  used  in  combat  aircraft.  Among  other  things,  the 
system  is  intended  to  aid  the  pilot  in  keeping  his/her  head  up  during  high-g  maneuvers.  One  possible 
advantage  of  such  a  system  that  is  related  to  SA  is  (p,  162),  "This  enables  the  pilot  to  maintain  a 
greatly  improved  level  of  awareness  of  his  surroundings." 

KEYWORDS:  aviation  (military),  pilot  aiding 
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43.  Dickson,  D.B,,  and  Hundley,  E.L.  (1992,  July).  Avoiding  not  so  friendly  fire. 
Military  Review,  72(7),  57-64. 

Fratricide  is  the  destruction  of  friendly  forces  by  friendly  forces.  Fratricide  has  always  occurred  durin. 
wars,  but  this  paper  argues  that  the  advances  in  range  and  lethality  of  weapons  combined  with  the 
faster  tempo  of  modem  combat  make  it  an  even  larger  problem  than  it  was  in  the  past.  This  point  was 
supported  by  the  35  deaths  and  72  woundings  that  occurred  during  Desert  Storm  that  were  due  to 
fratricide.  This  was  a  large  proportion  of  the  overall  casualties  that  occurred  during  that  war.  In  May 
1991  the  Army  formed  the  Combat  Identification  Task  Force  to  deal  with  the  problem  of  fratricide. 
Among  the  outcomes  of  this  task  force's  activities  was  the  conclusion  that  (p.  60),  "the  inability  to 
maintain  situational  awareness  in  combat  and  the  lack  of  positive  target  identification  capability  are  the 
major  contributors  to  fratricide."  Situational  awareness  was  defined  as  (p.  62),  "the  distributed 
knowledge  of  friendly  and  enemy  locations  in  the  context  ofMETT-T  (mission,  enemy,  terrain,  troops, 
and  time  available)."  Along  with  the  Combat  Identification  Task  Force,  the  Fratricide  Prevention  Task 
Force  was  formed  to  determine  how  to  integrate  fratricide  prevention  measures  into  the  Army.  The 
two  task  forces  have  issued  recommendations  that  involve  changes  in  doctrine,  acquisition  of  new 
material,  and  changes  in  training  to  increase  SA  and  reduce  the  potential  for  fratricide. 

KEYWORDS:  battlefield  SA,  definition,  fratricide,  target  identification,  training 


44.  Dornheim,  M.A.  (1986,  June  23).  Crew  situational  awareness  drives  avionics 
developments.  Aviation  Week  &  Space  Technoiogy,  724(25),  114-116. 

Dornheim  discusses  efforts  to  enhance  fighter  crew  situational  awareness  with  improved  avionics  on 
the  Air  Force  Advanced  Tactical  Fighter.  Avionics  which  incorporate  coordinated  sensors,  order  large 
quantities  of  information,  and  use  displays  keyed  to  natural  human  perception  (such  as  sound)  are  being 
developed.  Dornheim  states  that  key  challenges  include  finding  targets  while  minimizing  emissions, 
identi^ng  friend  or  foe,  and  displaying  this  information.  Future  "supercockpit"  features  under 
consideration  include  three  dimensional  presentation  of  information,  control  by  eye/voice,  rapid 
control/display  reconfiguration,  and  monitoring  of  pilot  condition.  Dornheim  also  briefly  discusses  a 
Stereographies  Corporation  three-dimensional  display  and  helmet  mounted  displays. 

KEYWORDS:  aviation  (military),  avionics,  display  design,  helmet-mounted  displays 
(HMDs),  pilot  aiding 
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45.  Dyck,  J.L.,  and  Gilson.  R.D  (1994).  Situation  awareness  in  marginal  weather 
conditions:  Do  graphical  presentations  help?  in  R.D.  Gilson,  D.J.  Garland,  and 
J.M.  Koonce  (Eds.),  Situational  awareness  in  complex  systems:  Proceedings  of  a 
CAHFA  Conference  (pp.  251-259).  Daytona  Beach,  FL:  Embry-Riddle  Aeronautical 
University  Press. 

In  this  project,  SA  is  seen  as  a  contributor  to  the  decision  making  process  of  a  pilot  faced  with  marginal 
weather  conditions.  For  most  General  Aviation  pilots,  weather  information  is  accessed  through  a 
textually-based  system.  The  authors  suggest  that  a  graphically-based  system  might  improve  the  SA  for 
weather  conditions.  This  was  tested  in  an  experiment.  The  flight  decisions  and  rationale  of  General 
Aviation  pilots  that  were  provided  with  either  textual  or  graphical  weather  information  were  compared 
to  the  decisions  of  an  expert  group  of  pilots.  The  decision  and  reasoning  of  the  subject  pilots  did  differ 
from  the  expert  pilots,  showing  an  effect  of  expertise.  However,  only  a  few  marginal  effects  were 
associated  with  the  format  of  the  weather  information.  Nevertheless,  the  authors  contend  that  the 
proper  use  of  graphical  weather  information  may  yet  prove  to  be  of  value,  and  make  suggestions  for 
future  research  based  on  their  experiences  with  this  project. 

KEYWORDS;  aviation  (civilian),  cognitive  processes,  decision  making,  display  design, 
expertise 


46.  Edens,  E.S.  (1991,  April).  Individual  differences  underlying  pilot  cockpit  error 
(Tech.  Report  No.  DOT/FAA/RD-91-13).  Washington,  DC:  US  Department  of 
Transportation.  (AD-A236107) 

This  paper  is  a  doctoral  dissertation.  The  question  studied  is  whether  personality,  attitudes,  and 
cognitive  ability  impact  the  likelihood  of  pilot  error.  Edens  defines  S  A  as  "the  accurate  perception  of 
the  factors  and  conditions  affecting  the  aircraft  and  the  flight  crew"  (p.7).  She  predicted  that  SA, 
psychological  stress,  and  hazardous  thought  patterns  are  cognitive  processes  which  act  as  mediating 
variables  between  individual  differences  (to  include  achievement  motivation,  vulnerability,  and 
extroversion)  and  pilot  error.  A  model  is  presented  outlining  this  relationship.  Student  helicopter 
pilots  (n=3 12)  served  as  subjects  in  this  study.  Measures  of  pilot  error  were  made  by  instructor  pilots 
and  evaluation  pilots.  Results  showed  that  only  pilot  situational  awareness  and  psychological  stress 
levels  predicted  the  frequency  of  cockpit  error.  Individual  differences  as  predictors  were  a 
disappointment. 

KEYWORDS:  cognitive  processes,  definition,  helicopter,  individual  differences, 

measurement,  performance-based  measures,  pilot  error,  subjective  measures 
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47.  Emerson,  T.J.,  Reising,  J.M.,  and  Britten-Austin,  H.G.  (1987).  Workload  and 
Situation  awareness  in  future  aircraft  (SAE  Technical  Paper  Series  No.  871803). 
Warrendale,  PA:  Society  of  Automotive  Engineers. 

This  article  discusses  the  potential  development  of  an  artificially  intelligent  Electronic  Crewmember 
(EC)  for  future  generations  of  fighter  aircraft.  The  need  for  real-time  input  on  the  pilot's  state  of 
workload  and  SA  into  the  EC  is  highlighted.  Physiological  indices  and  performance  profiles  are 
discussed  as  means  of  meeting  this  need.  The  authors  address  the  issue  of  defining  SA  (p.  4): 
"Situation  awareness  (SA)  can  be  defined  as  the  crew's  knowledge  of  both  the  internal  and  external 
states  of  the  aircraft,  as  well  as  the  environment  in  which  it  is  operating."  Among  the  ways  that  EC 
will  help  the  pilot  maintain  SA  are  the  off-loading  of  many  monitoring  responsibilities,  fusing  data  prior 
to  presentation,  allowing  high-level  command  communication  with  aircraft  systems,  mission  preview, 
and  real-time  reallocation  of  tasks. 

KEYWORDS:  aviation  (military),  avionics,  decision  aiding,  definition,  measurement, 
performance-based  measures,  physiological  measures,  pilot  aiding, 
workload 


48.  Emery,  C.D.,  and  Holding,  D.H.  (1993).  Practice  effects  on  the  Wombat  device.  In 
Proceedings  of  the  Seventh  International  Symposium  on  Aviation  Psychoiogy 
(Volume  1,  pp.  394-397).  Columbus:  The  Ohio  State  University. 

This  research  investigated  S  A  as  a  generic  skill.  A  testing  device  called  the  Wondrous  Original  Method 
for  Basic  Airmanship  Testing  (WOMBAT)  was  used  as  the  criterion.  The  WOMBAT  is  a  desktop 
testing  device  that  presents  a  tracking  task  along  with  a  set  of  selected  time-shared  tasks.  Two  studies 
were  conducted.  In  the  first  study,  WOMBAT  was  used  to  assess  a  group  of  20  commercial  airline 
pilots.  Ratings  of  each  of  the  pilots  on  basic  aviation  skills  and  cockpit  resource  management  skills 
were  collected  fi’om  the  pilots'  fleet  chief  pilots.  Also,  records  of  flight  training  checks  were  assessed. 
Correlation  among  all  of  the  measures  found  little  relationship  between  the  WOMBAT  measure  and 
any  of  the  other  measures  of  flying  proficiency.  In  the  second  study,  the  WOMBAT  performance  of 
pilots  was  compared  to  that  of  college  students.  Differences  in  the  learning  curves  of  the  two  groups 
were  observed.  In  the  discussion  the  authors  present  possible  reasons  for  the  lack  of  association 
between  WOMBAT  performance  and  the  measures  of  flight  proficiency.  They  also  suggest  that  the 
learning  curve  differences  observed  in  study  two  might  be  a  useful  selection  tool. 

KEYWORDS:  aviation  (civilian),  cognitive  processes,  expertise,  individual  differences, 
measurement,  performance-based  measures 
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49.  Endsley,  M.R.  (1987).  The  application  of  human  factors  to  the  development  of 
expert  systems  for  advanced  cockpits.  In  Proceedings  of  the  Human  Factors 
Society  31st  Annual  Meeting  (Volume  2,  pp,  1388-1392).  Santa  Monica,  CA: 
Human  Factors  Society. 

This  article  discusses  the  potential  impact  of  expert  systems  in  advanced  cockpits.  Endsley  suggests 
that  evaluation  of  automation  incorporating  expert  systems  must  include  measurement  of  four 
attributes:  workload,  situation  awareness,  performance,  and  acceptance.  Regarding  SA,  she 
specifically  mentions  the  Situation  Awareness  Global  Assessment  Technique  (S  AGAT)  as  a  potential 
measure  of  the  effect  of  a  new  cockpit  system  on  SA. 

KEYWORDS:  aviation  (military),  avionics,  measurement,  memory  probe  measures,  pilot 
aiding,  SAGAT,  workload 


50.  Endsley,  M.R.  (1988a).  Design  and  evaluation  for  situation  awareness 

enhancement.  In  Proceedings  of  the  Human  Factors  Society  32nd  Annual 
Meeting  (Volume  1,  pp.  97-101).  Santa  Monica,  CA:  Human  Factors  Society. 

This  paper  defines  situation  awareness  as  (p.  97),  "the  perception  of  the  elements  in  the  environment 
within  a  volume  of  time  and  space,  the  comprehension  of  their  meaning,  and  the  projection  of  their 
status  in  the  near  future."  Endsley  uses  this  definition  consistently  throughout  her  work.  She  places 
SA  in  the  framework  of  human  information  processing  models  with  a  grapliic  "aircrew  decision  model," 
and  with  a  figure  which  depicts  information  processing,  memory  structures,  and  attention  and  decision 
making,  entitled  "mechanisms  of  situation  awareness."  These  structures  are  discussed  in  detail. 

Thirteen  design  guidelines  are  listed  for  use  in  maximizing  the  SA  of  a  system,  and  then  the  Situation 
Awareness  Global  Assessment  Technique  (SAGAT),  a  technique  for  evaluating  pilot  situation 
awareness  in  simulators,  is  described.  Endsley  states  that  the  primaiy  limitation  of  SAGAT  is  that  the 
simulation  must  be  halted  to  collect  the  data,  and  then  lists  several  advantages  of  SAGAT. 

KE\"WORDS:  attention,  aviation  (military),  cognitive  processes,  decision  making, 

definition,  design  guidelines,  measurement,  memory  probe  measures,  pilot 
aiding,  SAGAT 


51.  Endsley,  M.R.  (1988b).  Situation  awareness  global  asseijsment  technique 

(SAGAT).  In  Proceedings  of  the  IEEE  1988  National  Aerospace  and  Electronics 
Conference  -  NAECON  1988  (Volume  3,  pp.  789-795).  New  York:  Institute  of 
Electricai  and  Electronics  Engineers. 

In  this  paper,  Endsley  mainly  focuses  on  the  SAGAT  methodology,  and  discusses  studies  undertaken 
to  validate  the  technique.  The  definition  of  SA  is  taken  a  step  further,  introducing  the  concept  of  SA 
zones  of  interest,  which  are  concentric  rings  representing  areas  of  immediate,  intermediate,  and 
long-term  interest  to  the  pilot.  SA  measurement  is  considered  from  a  viewpoint  where  the  goal  is  to 
evaluate  whether  SA  is  better  in  one  system  than  in  another.  Subjective,  physiological,  and 
questionnaire  methods  of  measuring  SA  are  critiqued,  and  then  Endsley  presents  the  SAGAT  approach 
in  detail.  A  paper  and  pencil  version  of  SAGAT  was  employed  in  a  1987  pilot  study;  this  work  is 
discussed,  along  with  another  study  in  whicn  SAGAT  was  used,  Endsley  con'^ludes  by  projecting 
future  growth  and  utilization  of  SAGAT. 

KEYWORDS:  definition,  measurement,  memory  probe  measures,  physiologicai  measures, 
questionnaire/survey  data,  retrospective  measures,  SAGAT,  subjective 
measures 


52.  Endsley,  M.R.  (1990a,  April).  A  methodology  for  the  objective  measurement  of 
pilot  situation  awareness.  In  AGARD-CP-478,  Situational  Awareness  in  Aerospace 
Operations  (pp.  1-1  to  1-9).  Neuilly  Sur  Seine,  France:  Advisory  Group  for 
Aerospace  Research  &  Development.  (AD-A223939) 

This  paper  begins  with  a  listing  of  areas  where  research  is  ongoing  in  situation  awareness;  sensor 
technology,  advanced  displays  and  controls,  and  automation  of  pilot  tasks  are  included  in  this  list. 
Endsley  states  that  researchers  have  been  hampered  by  lack  of  a  common,  consistent  definition  of  SA 
and  by  the  lack  of  an  objective  technique  for  evaluating  competing  design  concepts.  There  is  a 
comprehensive  section  which  breaks  down  the  elements  of  her  proposed  definition  of  SA,  and  a 
discussion  on  cognitive  foundations  of  SA.  Th^  SAGAT  procedure  is  described,  along  with  two 
studies  utilizing  the  technique,  one  on  memory  decay  and  one  dealing  with  frequency  and  duration  of 
halts  to  the  simulation  when  SAGAT  data  is  being  collected.  This  paper  concludes  by  suggesting 
research  in  the  areas  of  design,  training,  and  construct  exploration. 

KEYWORDS:  aviation  (military),  cognitive  processes,  combat  simulation,  definition, 
measurement,  memory  probe  measures,  performance-based  measures, 
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53.  Endsley,  M.R.  (1990b).  Predictive  utility  of  an  objective  measure  of  situation 
awareness.  In  Proceedings  of  the  Human  Factors  Society  34th  Annual  Meeting 
(Volume  1,  pp.  41-45).  Santa  Monica,  CA:  Human  Factors  Society. 

T)us  paper  describes  results  of  a  study  undertaken  to  compare  performance  of  pilots  in  a  simulated 
air-to-air  engagement  with  objective  SAGAT  measures  of  SA.  Performance  was  measured  by  air-to-air 
kills  and  losses  during  the  simulation,  which  pitted  two  teams  of  pilots  against  each  other  with  specific 
(but  different)  goals  and  rules  of  engagement  on  each  side.  The  study  found  that  one  team  was  limited 
by  the  rules  of  engagement  and  by  equipment  and  thus  was  less  able  to  take  advantage  of  its  SA  in 
terms  of  kill  ratio.  The  other  team  displayed  a  significantly  better  likelihood  of  later  killing  an  aircraft 
when  they  had  SA  of  that  aircraft.  Lower  SA  did  not  always  accompany  poor  performance;  pilots 
possibly  were  aware  that  their  S  A  was  poor  and  acted  more  conservatively,  according  to  Endsley.  She 
also  proposed  reasons  a  why  there  were  differences  in  SA  among  pilots.  Endsley  concludes  that  the 
predictive  relationship  b,  ,een  SA  as  measured  by  SAGAT  and  pilot  performance  is  supported  by  this 
study. 

KEYWORDS;  air-to-air  mission,  aviation  (military),  cognitive  processes,  combat 
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54.  Endsley,  M.R.  (1991a).  Situation  awareness  in  an  advanced  strategic  mission 
(Tech.  Report  No.  AL-TR-1 991 -0083).  Wright-Fatterson  Air  Force  Base,  OH:  Air 
Force  Systems  Command.  (AD-B161348) 

This  report  includes  extensive  discussion  of  the  issues  associated  with  situation  awareness.  As  in  other 
papers,  the  author  defines  situation  awareness  as  (p.  7),  "the  perception  of  the  elements  in  the 
environment  within  a  volume  of  time  and  space,  the  comprehension  of  their  meaning,  and  the  projection 
of  their  status  in  the  near  future,"  The  author  is  careful  to  distinguish  situation  awareness  from 
decision  making.  Situation  awareness  is  considered  to  be  the  input  to  the  decision  making  process. 

The  author  also  extends  the  definition  of  .situation  awareness  to  take  into  account  crew  situation 
awareness  (p.  9):  "Comprehensive  crew  SA  can  be  conceived  of  as  'the  degree  to  which  every  crew 
member  possesses  the  SA  required  for  his  position.'"  The  discu.ssion  of  theoretical  underpinning  of 
situation  awareness  emphasizes  memory  processes  (including  sensory  memory,  working  memory 
capacity,  knowledge  in  long-term  memory,  etc)  and  attention.  Based  on  this  discussion,  13 
recommendation  for  system  design  to  optimize  situation  awareness  are  presented  and  the  possible  role 
of  individual  differences  is  discussed.  Potential  situation  awareness  measurement  techniques  are 
discussed.  Subjective  r  atings  are  considered  problematic  because  the  crewmember  may  not  realize  that 
there  is  important  information  that  he  is  unaware  of,  and  also  his  ratings  may  be  biased  by  the  outcome 
of  the  mission.  Physiological  techniques  are  considered  desirable,  but  unproven.  Questionnaires  are 
suspect  due  to  the  limitations  of  human  memory.  The  author  argues  that  the  best  measurement 
approach  is  represented  by  the  Situation  Awareness  Global  Assessment  Technique  (SAGAT).  SAGAT 
involves  randomly  stopping  a  simulated  mission  and  asking  a  series  of  questions  to  ascertain  the  state 
of  the  crewmembers  situation  awareness.  The  design  of  a  SAGAT  analysis  of  a  strategic  mission  is 
described  in  great  detail.  No  data  are  presented. 
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55.  Endsley,  M.R.  (1991b).  Situation  awareness  in  dynamic  systems.  In  Designing 
for  Everyone:  Proceedings  of  the  11th  Congress  of  the  International  Ergonomics 
Association  (Volume  1,  pp.  801-803).  Paris,  France:  Taylor  &  Francis. 

This  paper,  along  with  4  other  S  A  related  papers  presented  at  the  Congress,  were  reprinted  in  Taylor 
(1991)  and  is  reviewed  under  that  listing. 

KEYWORDS:  measurement,  memory  probe  measures,  SAGAT 


56.  Endsley,  M.R.  (1993a).  Situation  awareness  and  workload:  Flip  sides  of  the  same 
coin.  In  Proceedings  of  the  Seventh  International  Symposium  on  Aviation 
Psychology  (Volume  2,  pp.  906-911).  Columbus:  The  Ohio  State  University. 

The  author  examines  the  issue  of  the  relationship  between  mental  workload  and  situation  awareness. 
Her  hypothesis  is  stated  as,  "SA  and  workload,  although  inter-related,  are  hypothesized  to  be 
essentially  independent  constructs."  The  concepts  are  assumed  to  be  inter-related  because  situation 
awareness  involves  mental  information  processing  and  this  contributes  to  mental  workload.  So,  in 
many  environments,  the  investment  of  additional  mental  workload  may  be  associated  with  increasing 
situation  awareness.  However,  a  person  operating  in  a  familiar,  well-understood  environment  might 
have  excellent  situation  awareness  combined  with  very  low  mental  workload.  Or  alternatively,  a 
person  may  be  working  very  hard  in  an  attempt  to  understand  a  situation,  but  fail.  The  relationship 
between  workload  and  situation  awareness  was  tested  by  comparing  subjective  workload  ratings  to  a 
situation  awareness  metric  collected  during  simulated  air  combat.  The  SWAT  procedure  was  used  to 
assess  mental  workload,  and  SAGAT  was  used  to  assess  situation  awareness.  Six  experienced  former 
military  pilots  served  as  the  subjects.  Overall,  there  was  no  correlation  between  SWAT  ratings  and 
SAGAT  scores,  supporting  the  hypothesis.  However,  evaluation  of  the  individual  pilot  data  found  that 
two  of  the  pilots  correlated  higher  workload  with  lower  situation  awareness.  Despite  these  two  pilots 
showing  a  relationship,  the  overall  conclusion  was  that  the  hypothesized  independence  was  supported. 
The  author  also  suggests  that  this  independence  might  be  inconsistent  with  attempts  of  some  metric 
techniques  (e.g.,  SART)  to  combine  elements  of  both  concepts. 

KEYWORDS:  aviation  (military),  cognitive  processes,  combat  simulation,  measurement, 
memory  probe  measures,  SAGAT,  SART,  subjective  measures,  workload 
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57.  Endsley,  M.R.  {1993b).  A  survey  of  situation  awareness  requirements  in  air-to-air 
combat  fighters.  The  International  Journal  of  Aviation  Psychology,  3,  157-168. 

The  author  reiterates  her  previous  definition  of  SA  (see  Endsley,  1988a).  The  paper  then  describes  a 
procedure  for  determining  the  specific  SA  needs  consistent  with  the  SA  definition  for  a  given  task 
domain.  Air-to-air  combat  is  utilized  as  an  example  domain  to  illustrate  the  procedures.  The  procedure 
consists  of  three  steps:  One,  the  author  conducted  unstructured  interviews  with  subject-matter  experts 
to  determine  their  thoughts  about  the  nature  of  SA  and  its  relationship  with  mission  performance. 

Two,  the  author  conducted  a  goal-directed  task  analysis  by  observing  simulated  air-to-air  combat.  The 
observations  were  used  to  identify  primary  goals,  subgoals,  and  SA  needs  for  the  various  segments  of 
air-to-air  combat.  These  data  were  represented  in  tree-diagrams  that  were  then  critiqued  by 
subject-matter  experts.  Third,  based  on  the  results  of  the  first  two  phases,  a  structured  questionnaire 
was  constructed  and  distributed  to  another  set  of  subject-matter  experts  to  elicit  criticality  judgments. 
The  results  of  these  judgments  are  tabulated  in  the  paper.  Tl  Js  paper  will  be  especially  interesting  for 
researchers  engaged  in  evaluating  SA  in  an  air-to-air  domain.  However,  the  procedure  could  also  be 
used  in  other  domains  to  specify  SA  needs. 

KEYWORDS:  air-to-air  mission,  aviation  (military),  cognitive  processes,  definition, 

expertise,  measurement,  questionnaire/survey  data,  subjective  measures 


58,  Endsley,  M.R.  (1994a).  Situation  awareness  in  dynamic  human  decision  making: 
Theory.  In  R.D.  Gilson,  D.J.  Garland,  and  J.M.  Koonce  (Eds.),  Situational 
awareness  in  complex  systems:  Proceedings  of  a  CAHFA  Conference  (pp.  27-58). 
Daytona  Beach,  FL:  Embry-Riddle  Aeronautical  University  Press. 

In  this  paper,  Endsley  presents  the  most  thorough  review  of  her  theoretical  views  on  SA  (to  date).  As 
a  definition  for  SA,  the  definition  from  Endsley  (1988a)  is  adopted.  She  suggests  that  SA  will  be  a 
vital  theoretical  construct  for  understanding  and  improving  performance  in  a  wide  variety  of  operational 
settings.  The  theoretical  approach  selected  for  addressing  SA  is  James  Reason's  concept  of  a 
framework  model.  In  this  approach  the  goal  is  to  synthesize  information  from  a  wide  variety  of  setting 
and  theories  into  a  good  descriptive  model.  As  background  for  her  framework  model,  Endsley  states 
that  (p.  34):  "In  combination,  the  mechanisms  of  short  term  sensory  memory,  perception,  working 
memory  and  long  term  memory  form  the  basic  structures  on  which  SA  is  based."  The  contribution  and 
interaction  of  these  processes  are  described.  Attention  is  also  discussed  as  an  important,  limiting 
component  of  SA  (p.  36);  "Because  the  supply  of  attention  appears  to  be  limited,  improvements  in  SA 
on  some  elements  may  mean  decrements  in  SA  on  others  once  the  limit  is  reached.  And  this  limit  may 
occur  rather  quickly  in  complex  environments. "  The  paper  also  discusses  the  possible  roles  of  mental 
models  in  SA,  the  role  of  SA  in  errors,  and  Iiow  the  SA  concept  can  be  extended  to  team  environments. 
The  paper  concludes  with  a  discussion  of  future  direction  for  SA  research  in  human-machine  interface 
design,  training,  individual  differences,  and  theoretical  constructs. 

KEYWORDS;  attention,  cognitive  processes,  decision  making,  definition,  expertise, 
mental  models,  team  SA,  training,  workload 
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59.  Endsley,  M.R.  (1994b).  Situation  awareness  in  dynamic  human  decision  making: 

Measurement.  In  R.D.  Gilson,  D.J.  Garland,  and  J.M.  Koonce  (Eds.),  Situational 

awareness  in  complex  systems:  Proceedings  of  a  CAHFA  Conference  (pp.  79-97). 

Daytona  Beach,  FL:  Embry-Riddle  Aeronautical  University  Press. 

This  article  discusses  the  potential  uses  of  several  approaches  for  measuring  SA.  As  a  definition  for 
SA,  the  definition  from  Endsley  (1988a)  is  adopted.  Physiological  measurement  is  seen  as  having 
limited  usefulness  (p.  8 1);  "These  techniques  will  allow  researchers  to  determine  if  elements  in  the 
environment  are  perceived  and  processed  by  subjects,  but  do  not  allow  a  determination  of  how  much 
information  remains  in  memory,  if  the  information  is  registered  correctly  in  the  mind,  or  what 
comprehension  the  subject  has  of  those  elements."  Performance  measures  are  divided  into  three  main 
categories:  Global  measures.  External  task  measures,  and  Imbedded  task  measures,  All  are  seen  as 
having  the  advantages  of  being  objective  and  usually  non-intrusive  to  assess.  But,  global  measures  of 
performance  are  seen  as  having  problems  with  diagnosticity  and  sensitivity.  External  task  measures  are 
criticized  as  being  very  intrusive  and  based  on  questionable  assumptions.  Imbedded  task  measures 
suffer  from  being  overly  focused  on  a  small  subset  of  the  operator's  subtasks,  and  potentially 
insensitive  to  changes  in  SA  elsewhere  in  the  system.  The  next  category  of  SA  measurement  is 
subjective  measures,  which  is  divided  into  two  subcategories;  Self-rating,  and  Observer-rating. 

Endsley  identifies  several  sources  of  contamination  of  SA  self-ratings  (e.g.,  SART  and  SA-SWORD), 
and  suggests  that  (p.  83),  "subjective  self-ratings  of  SA  most  likely  convey  a  measure  of  the  subjects' 
confidence  level  regarding  that  SA."  It  is  suggested  that  observer-ratings  of  SA  might  be  limited  by 
the  observers'  lack  of  access  to  the  subjects'  concept  of  the  situation.  The  final  category  of  SA 
measurement  is  Questionnaires,  which  is  subdivided  into;  Post-test,  On-line,  and  Freeze  Technique. 
Questionnaires  are  seen  as  good  for  obtaining  detailed  information  about  the  subject's  SA,  but  the 
post-test  version  is  seen  as  being  too  limited  by  tendencies  "to  over  generalize,  over  summarize  and 
over  rationalize"  (p.  85)  about  past  mental  events.  On-line  vocalization  was  seen  as  too  intrusive.  As  a 
solution  to  the  difficulties  with  the  post-test  or  on-line  approaches,  the  use  of  a  freeze  technique  (such 
as  SAGAT)  was  advocated.  In  a  freeze  procedure,  the  simulation  is  frozen  at  randonJy  selected 
moments  and  the  subject  answers  questions  about  their  current  perceptions  of  the  situation.  A 
potential  difficulty  with  this  approach  is  the  perceived  intrusiveness  of  stopping  the  simulation.  Two 
studies  are  reviewed  that  suggest  that  the  SAGAT  technique  can  be  sensitive  without  be  intrusive. 

KEYW'ORDS:  definition,  measurement,  memory  probe  measures,  performance-based 

measures,  physiological  measures,  questionnaire/survey  data,  retrospective 
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60.  Fracker,  M.L.  (1988a).  A  schema-based  model  of  situation  awareness:  Implications 
for  measuring  situation  awareness.  In  Second  Annual  Workshop  on  Space 
Operations,  Automation,  and  Robotics  -  {SOAR  ’88)  (Tech.  Report  No. 
NASA-CP-3019,  pp.  227-231).  Washington,  DC:  National  Aeronautics  and  Space 
Administration. 

The  author  is  primarily  concerned  with  addressing  two  issues;  the  definition  of  SA,  and  modeling  how 
SA  is  maintained.  To  define  SA,  he  starts  by  addressing  the  issue  of  defining  a  situation  (p.  227):  "I 
define  a  situation  to  be  a  set  of  processes  that  control  events  in  the  environment....  A  situation,  then, 
can  be  defined  at  variou.s  levels  of  abstraction.  At  the  highest  level,  the  situation  might  be  defined  in 
terms  of  the  goals  of  the  human  participants.  At  the  lowest  level,  the  situation  may  be  defined  in  terms 
of  the  momentary  states  of  objects  in  the  environment.  In  between  these  two  extremes,  the  situation 
may  be  defined  in  terms  of  the  organizations,  functions,  or  processes  that  translates  goals  into  states." 

SA  is  then  defined  in  terms  of  this  viewpoint  of  a  situation  (p.  227),  "Situation  awareness,  therefore, 
can  be  defined  partly  as  the  knowledge  that  results  when  attention  is  allocated  to  the  environment  at 
one  or  more  levels  of  abstraction."  The  model  for  understanding  the  cognitive  processes  of  SA  is  based 
upon  the  implications  of  schema  theory.  The  issues  of  how  the  schema  control  the  allocation  of 
attention  and  are  influenced  by  the  limits  of  working  memory  capacity  are  also  discussed.  One  negative 
aspect  of  the  use  of  schema  is  said  to  be  the  propensity  for  schema-based  information  processing  to  be 
affected  by  biases.  The  author  suggests  that  the  fact  that  SA  represents  a  mixing  of  prior  knowledge 
and  current  perception  produces  challenges  for  any  memory  based  measurement  technique.  As  a  final 
conclusion,  the  author  states  (p.  230),  "the  model  suggests  that  measuring  the  load  on  working 
memory  imposed  by  situation  assessment  may  be  as  important  as  measuring  SA  itself"  In  other 
words,  high  levels  of  mental  workload  are  seen  as  a  threat  to  the  maintenance  of  good  SA. 

KEYWORDS;  attention,  cognitive  processes,  expertise,  measurement,  memory  probe 

measures,  retrospective  measures,  schema,  subjective  measures,  workload 


61.  Fracker,  M.L.  {1988b).  A  theory  of  situation  assessment:  Implications  for 
measuring  situation  awareness.  In  Proceedings  of  the  Human  Factors  Society 
32nd  Annuai  Meeting  (Volume  1,  pp.  102-106).  Santa  Monica,  CA:  Human  Factors 
Society. 

In  this  paper,  Fracker  defines  SA  as  "...the  knowledge  that  results  when  attention  is  allocated  to  a  zone 
of  interest  at  a  level  of  abstraction."  He  goes  on  to  explain  this  definition  in  detail,  and  then  describes  a 
model  of  SA  based  on  knowledge  structures,  or  "schemata."  He  proposes  that  pilots  recognize  patterns 
of  sensoi7  information  and  apply  established  schemata  to  fill  in  the  details  of  the  situation.  If  an 
appropriate  schema  is  not  available  or  is  not  identified,  the  load  on  the  pilot's  working  memory  is 
increased  greatly  by  the  alternative  processes  required.  Fracker  discusses  five  cognitive  biases  inherent 
in  this  schemata  theory.  Memory-probe  measures  of  SA  and  some  problems  associated  with  them  are 
reviewed.  In  closing,  Fracker  proposes  a  secondary  memory  span  task  for  measuring  the  load  on 
working  memory  associated  with  SA. 

KEYWORDS:  attention,  cognitive  processes,  definition,  expertise,  measurement,  memory 
probe  measures,  retrospective  measures,  schema,  subjective  measures 
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62.  Fracker,  M.L.  (1989).  Attention  allocation  in  situation  awareness.  In  Proceedings 
of  the  Human  Factors  Society  33rd  Annual  Meeting  (Volume  2,  pp.  1396-1400). 
Santa  Monica,  CA:  Human  Factors  Society. 

The  paper  presents  research  investigating  the  impact  of  limited  attentional  capacity  on  situation 
awareness  and  the  utility  of  memory  probes  as  situation  awareness  metrics.  Subjects  were  given  a 
"God's-eye"  view  of  a  simulated  air  battle  involving  one  aircraft  under  their  control  and  a  wing-man. 

An  additional  five  aircraft  were  always  present,  but  the  proportion  of  enemies  to  neutral  was  varied. 
Also,  on  some  trials  the  identity  of  the  other  aircraft  could  change  during  the  trial.  Based  on  the 
assumption  that  attentional  capacity  is  limited,  it  was  predicted  that  situation  awareness  would  be 
unequally  distributed  over  the  different  aircraft  types.  In  specific,  spatial  localization  of  neutrals  was 
expected  to  be  less  accurate  than  enemies.  However,  since  aircraft  identity  presumably  had  to  be 
known  prior  to  allocating  attention,  it  was  expected  that  identifying  identity  would  be  unaffected  by 
the  Neutral/Enemy  distinction.  These  hypotheses  were  tested  via  memory  probe  situation  awareness 
metrics.  At  random  times,  the  trial  stopped  and  subjects  were  asked  to  identify  one  aircraft  (Friend, 
Enemy,  or  Neutral)  and  show  the  coirect  location  for  another  aircraft.  Results  were  consistent  with  the 
expected  allocation  of  attention.  The  author  concluded  that  the  results  supported  the  viability  of 
memory  probes  as  an  approach  for  situation  awareness  assessment. 

KEYWORDS:  attention,  cognitive  processes,  measurement,  memory  probe  measures, 
target  identification 


63.  Fracker,  M.L.  (1990,  April).  Attention  gradients  in  situation  awarenes.s.  In 

AGARD-CP-478,  Situational  Awareness  in  Aerospace  Operations  (pp.  6-1  to  6-10). 
Neuilly  Sur  Seine,  France:  Advisory  Group  for  Aerospace  Research  & 

Development.  (AD-A223939) 

This  paper  describes  a  series  of  four  studies  undertaken  with  the  objective  of  supporting  a  theory  of  a 
limited  supply  of  attention  by  demonstrating  reallocation  of  that  attention.  Specifically,  he  proposes 
that  if  there  is  a  finite  amount  of  attention,  then  it  will  most  likely  be  reallocated  to  threatening  targets 
(enemy  aircraft)  as  the  total  number  of  aircraft  in  a  simulation  increases.  Fracker  found  support  for  this 
theory.  He  used  spatial  awareness  and  ffiend/foe/neutral  probe  reaction  time  as  measures  of  attention 
allocation.  He  describes  a  phenomenon  whereby  we  tend  to  ignore  neutral  information,  in  this  case 
represented  by  neutral  aircraft,  attending  instead  to  targets  which  are  threatening  to  or  supportive  of 
our  own  mission, 

KEYWORDS:  attention,  cognitive  proce.sses,  measurement,  memory  probe  measures 


81 


64,  Fracker,  M.L.  (1991a,  October).  Measures  of  situation  awareness:  An 

experimentai  evaiuation  (Tech.  Report  No.  AL-TR-1991-0127).  Wright-Patterson 
Air  Force  Base,  OH:  Armstrong  Laboratory.  (AD-A262732) 

This  report  describes  a  complex  set  of  studies  conducted  with  several  goals  in  mind.  Mainly,  Fracker 
sought  to  gather  reliability  data  and  establish  criterion  and  construct  validity  for  a  set  of  SA  measures. 
The  measures,  both  implicit  and  explicit,  were  designed  to  support  a  multiple  resource  theory  of 
processing.  Three  studies  were  conducted.  Fracker's  two  hypotheses  were  (1)  object  locations  and 
identities  are  maintained  in  separate  working  memories,  and  (2)  these  two  working  memoiies  have 
separate  processing  resources,  so  that  increasing  processing  difficulty  in  one  has  no  effect  on  the  other. 
Subjects  viewed  several  objects  on  a  screen,  objects  color  coded  as  to  whether  they  were  friend,  foe,  or 
neutral.  The  implicit  measure  was  "envelope  sensitivh}',"  a  measure  of  how  well  subjects  completed  a 
task  involving  destroying  the  enemy  targets  on  the  .screen.  The  explicit  measures  were  derived  by 
freezing  the  screen  during  the  task  and  asking  probe  questions  about  one  of  the  objects'  identity  or 
location.  The  screen  also  contained  "death  zones"  which  subjects  had  to  avoid,  creating  another 
measure  of  performance  called  avoidance  failures.  Fracker  looked  at  test-retest  reliability  by  running 
the  subjects  over  two  separate  sessions.  The  reliability  measures  proved  inconclusive,  in  that  none  of 
the  measures  were  highly  reliable  In  assessing  criterion  validity,  subjects'  "kill  probability"  was  the 
criterion  measure  against  which  identity  and  location  probe  measures  were  compared.  Identity  accuracy 
and  latency  validity  coefficients  were  small,  and  the  author  attributes  this  to  the  low  reliability.  Object 
location  error  was  not  a  good  predictor.  The  best  predictor  was  the  avoidance  failure  measure,  which  is 
described  as  more  of  a  workload  measure  than  a  SA  measure.  In  examining  construct  validity  via  a 
"separate  resources"  theory,  Fracker  found  that  spatial  and  verbal  capabilities  combined  to  overload 
working  memory'.  He  therefore  concludes  that  either  the  validity  of  these  measures  should  be 
questioned,  or  individuals  possess  just  one  common  processing  resource. 

KEYWORDS:  attention,  cognitive  processes,  implicit  measures,  measurement,  memory 
probe  measures,  multiple  resource  theory,  reliability 


65.  Fracker,  M.L.  (1991b,  October).  Measures  of  situation  awareness:  Review  and 
future  directions  (Tech.  Report  No.  AL-TR-1991-0128).  Wright-Patterson  Air  Force 
Base,  OH:  Armstrong  Laboratory.  (AD-A262672) 

In  this  paper,  Fracker  discusses  definitions  and  examples  of  reliability  and  validity  (content,  criterion, 
and  construct)  as  they  pertain  to  measurement  of  S  A.  The  author  then  examines  several  types  of  SA 
measures  which  have  been  used  in  previous  research,  reviewing  how  various  authors  employed  these 
measures  and  assessing  them  in  terms  of  reliability  and  the  three  types  of  validity  mentioned  above. 
Explicit,  implicit,  and  subjective  rating  measures  are  all  treated  in  this  fashion.  This  is  a  good  paper  to 
consult  if  one  is  considering  how  to  implement  an  SA  metric  in  any  kind  of  laboratory  or  field  study. 

KEYWORDS:  implicit  measures,  measurement,  memory  probe  measures,  reliability, 
retrospective  measures,  review  paper,  subjective  measures 
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66.  Fracker,  M.L.,  and  Davis,  S.A.  (1991,  October).  Explicit,  implicit,  and  subjective 
rating  measures  of  situation  awareness  in  a  monitoring  task  (Tech.  Report  No. 
AL'TR-1 991 -0091).  Wright-Patterson  Air  Force  Base,  OH:  Armstrong  Laboratory. 
(AD-A262702) 

Subjects  monitored  a  display  on  which  six  colored  objects  moved  about  on  a  grid.  They  v«^atched  for  an 
unpredictable  flash  of  any  object  (an  embedded  signal  detection  measure  of  situation  awareness). 
Occasionally,  the  display  would  freeze  and  the  objects  would  disappear.  Subjects  would  then  answer 
memory  probes  concerning  the  color  and  location  of  selected  objects.  Subjective  ratings  were  also 
collected.  Results  suggested  that  the  flash  detection  and -color  memory  probes  were  effective  situation 
awareness  metrics.  The  location  probes  seemed  to  be  insensitive  measures  of  situation  awareness.  The 
subjective  ratings  appeared  to  be  useful,  but  the  authors  cautioned  that  (p.  23),  "subjective  measures 
may  be  measuring  subjects'  rational  inferences  about  their  SA  or  workload." 

KEYWORDS:  implicit  measures,  measurement,  memory  probe  measures, 

performance-based  measures,  subjective  measures,  workload 


67.  Fracker,  M.L.,  and  Vidulich,  M.A.  (1391).  Measurement  of  situation  awareness:  A 
brief  review.  In  Designing  for  Everyone:  Proceedings  of  the  1 1th  Congress  of  the 
International  Ergonomics  Association  (Volume  1,  pp,  795-797).  Paris,  France: 
Taylor  &  Francis. 

This  paper,  along  with  4  other  SA  related  papers  presented  at  the  Congress,  were  reprinted  in  Taylor 
(1991)  and  is  reviewed  under  that  listing. 

KEYWORDS:  implicit  measures,  measurement,  memory  probe  measures,  retrospective 
measures,  subjective  measures 


68.  Fulghum,  D.A.  (1993,  February  8).  Night-fighting  CAS  force  gains  preliminary 
approval.  Aviation  Week  &  Space  Technology,  138(6),  52-54. 

The  article  discusses  potential  modifications  to  F-16  aircraft  to  enhance  their  effectiveness  in  nighttime 
ground-  attack  missions.  As  an  alternative  to  a  head-steered  Forward-Looking  Infrared  (FLIR)  system 
the  Air  Force  suggested  upgrading  F-16s  already  equipped  with  the  Low- Altitude  Navigation  and 
Targeting  Infrared  (LANTIRN)  system.  In  part,  the  suggestion  appears  to  be  a  cost-savings  move. 
But,  it  is  also  suggested  that  (p.  54),  "The  F-16's  modifications  -  including  ...  night  vision  goggles 
and  global  positioning  system  (GPS)  -  give  the  ground  attack  pilot  better  situational  awareness." 

KEYWORDS:  air-to-ground  mission,  aviation  (military),  pilot  aiding 
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69.  Fulghum,  D.A.  (1993,  July  5).  Major  changes  planned  for  wild  weasel  force. 
Aviation  Week  &  Space  Technology,  139{^),  40-41. 

The  article  describes  proposed  future  developments  for  military  aircraft,  such  as  the  F-4,  assigned  to  the 
Suppression  of  Enemy  Air  Defenses  (SEAD)  mission.  Two  proposed  changes  are  expected  to  increase 
the  SA  of  the  SEAD  aircraft  crew.  The  first  is  communications  with  sensors  from  other  aircraft  or 
satellites.  The  second  proposed  change  is  the  addition  of  a  tactical  situation  display  for  presenting  the 
information. 

KEYWORDS:  air-to-ground  mission,  aviation  (military),  avionics,  data  fusion,  data  link, 
display  design,  pilot  aiding 


70.  Fulghum,  D.A,  (1993,  August  23).  Talon  Lance  gives  aircrews  timely  intelligence 
from  space.  Aviation  Week  &  Space  Technology,  739(8),  70-71. 

The  article  discusses  the  Talon  Lance  program.  This  program  is  aimed  at  developing  on-board 
computer  analysis  capabilities  to  analyze  in  real-time  the  intelligence  information  from  satellites,  other 
warplanes,  etc.  Currently,  such  information  is  hours  old  and  available  only  at  takeoff  with  in-flight 
updates  via  radio.  The  Talon  Lance  system  will  present  a  filtered  version  of  all  of  this  information  to 
the  pilot  during  the  mission.  Among  the  rationale  for  justifying  the  system  was  "extending  situational 
awareness"  (p.  71). 

KEYWORDS:  aviation  (military),  avionics,  data  fusion,  data  link,  pilot  aiding 


71.  Furness,  T.A.  (1986).  The  super  cockpit  and  its  human  factors  challenges.  In 
Proceedings  of  the  Human  Factors  Society  30th  Annual  Meeting  (Volume  1,  pp, 
48-52).  Santa  Monica,  CA:  Human  Factors  Society. 

This  article  presents  the  design  philosophy  behind  the  development  of  the  Super  Cockpit  research 
program.  The  Super  Cockpit  was  envisioned  as  a  generic  crew  station  that  would  use  virtual  visual, 
auditory,  and  tactile  displays  to  exploit  the  full  perceptual  capabilities  of  the  human  pilot.  Among  the 
anticipated  advantages  of  the  Super  Cockpit  was  one  directly  related  to  SA  (p.  51),  "the  display 
medium  truly  offers  a  port  for  the  avionics  to  communicate  a  spatial  awareness  to  the  pilot  in  all 
directions  and  in  three  dimensions."  Among  the  challenges  posed  to  the  human  factors  community  by 
the  Super  Cockpit  was  (p.  52),  "How  can  the  level  of  situation  awareness  conveyed  to  the  pilot  be 
assessed?" 

KE\"WORDS:  3-D  audio  displays,  display  design,  helmet-mounted  displays  (HMDs),  pilot 
aiding 
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72.  Garland,  D.J.,  and  Hopkin,  V.D.  (1994).  Controlling  automation  in  future  air  traffic 
control:  The  impact  on  situational  awareness.  In  R.D.  Gilson,  D.J.  Garland,  and 
J.M.  Koonce  (Eds.),  Situational  awareness  in  complex  systems:  Proceedings  of  a 
CAHFA  Conference  (pp.  179-197).  Daytona  Beach,  FL:  Embry-Riddle  Aeronautical 
University  Press. 

This  article  covers  much  of  the  same  ground  as  Hopkin  (1994)  in  relating  situation  awareness  to  air 
traffic  control.  However,  it  goes  into  more  detail  in  discussing  proposed  changes  in  the  automation 
supporting  air  traffic  controllers  and  the  possible  effects  on  the  air  traffic  controller's  SA. 

KEYWORDS;  a''"  traffic  control,  cognitive  processes,  individual  differences,  mental 
models,  training 


73.  Gibson,  C.P.,  and  Garrett,  A.J.  (1990,  April).  Towards  a  future  cockpit  -  The 
prototyping  and  pilot  integration  of  the  mission  management  aid  (MMA).  In 
AGARD-CP-478,  Situational  Awareness  in  Aerospace  Operations  (pp.  7-1  to  7-9). 
Neuilly  Sur  Seine,  France:  Advisory  Group  for  Aerospace  Research  & 
Development.  (AD-A223939) 

This  paper  was  presented  at  the  1989  AGARD  meeting  on  Situational  Awareness  in  Aerospace 
Operations.  Gibson  and  Garrett  define  SA  as  (p.  7-1)  "...the  pilot's  overall  appreciation  of  his  current 
'world, '...[implying]  both  sensory  processing  and  im'erencing  [from]  previous  knowledge  and 
experience."  Because  SA  is  a  Gestalt-type  concept,  "greater  than  the  sum  of  the  parts,"  the  authors 
propose  that  it  is  difficult  to  measure  and  "limited  in  its  utility  as  a  tool  to  predict  performance"  (p. 
7-1).  The  paper  describes  the  mission  management  aid  (MMA),  which  in  essence  provides  the  pilot 
with  a  plan  for  action  based  on  the  environmental  and  aircraft  information  available.  The  pilot  may 
accept  or  reject  this  plan,  or  may  query  the  MMA  for  further  underlying  information.  The  prototype 
MMA  is  designed  to  operate  in  an  air-to-ground  role,  and  to  interact  with  other  nearby  MMAs.  The 
paper  also  details  how  the  MMA  design  process  was  accomplished.  Rather  than  top-down  or 
bottom-up  design,  a  flexible  mixture  of  the  two  was  employed  with  a  heavy  emphasis  on  incorporating 
human  factors  aspects  early  in  the  design  process. 

KEYWORDS:  air-to-ground  mission,  attention,  aviation  (military),  avionics,  cognitive 
processes,  data  fusion,  definition,  pilot  aiding 
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74.  Goldworm,  M.,  and  Millar,  J.  (1993,  September).  Fast  mapping  for  cockpit 
displays.  Defense  Electronics,  25(10),  42-44. 

The  authors  suggest  that  dynamic,  flexible  map  displays  would  be  an  effective  means  for  improving 
pilot  SA.  They  contrast  two  technological  approaches  for  achieving  this  goal.  Many  map  displays  are 
based  on  bit-mapped  images  generated  by  scanning  paper  charts.  This  can  produce  a  high  resolution 
image  in  a  familiar  format  and  enjoys  certain  technological  advantages.  However,  the  authors  suggest 
that  an  alternative  approach  based  on  the  Vector  Product  Format  (VPF)  may  have  functional 
advantages  because  the  flexible  way  the  map  data  is  coded  allows  for  more  flexibility  in  display 
formats.  On  the  other  hand,  the  VPF  approach  currently  faces  technological  challenges.  The  authors 
conclude  (p.  44),  "Both  the  right  software  and  processing  hardware  are  necessary  to  achieve  the 
performance  and  flexibility  required  of  modern  weapon  systems.  Designers  of  mapping  and  situational 
awareness  systems,  whether  in  the  cockpit  or  on  the  ground,  have  to  understand  these  issues  and  plan 
for  them  in  system  development  efforts." 

KEYWORDS:  aviation  (military),  display  design,  pilot  aiding 


75.  Goodson,  W.L.,  Walters,  R.V.,  and  Stites,  R.L.  (1990).  An  artificial  neural 
network  for  the  estimation  of  tactical  situation  awareness  (Tech.  Report  No. 
USAFSAM-TP-90-9).  Brooks  Air  Force  Base,  TX:  Human  Systems  Division. 

(AD-B1 46397) 

The  goal  of  this  research  is  to  establish  a  framework  artificial  neural  network  (ANN)  from  which  to 
develop  an  SA  training  device  (in  the  Phase  II  SBIR).  Goodson  et  al.  have  proposed  a  conceptual 
framework  which  breaks  the  S  A  construct  down  into  a  progression  of  tactical  data  vectors  (tactical 
environment  data  e,  aircraft  environment  data  /,  pilot  SA  or  knowledge  k,  tactical  response  o,  and 
workload  w).  The  domain  chosen  for  development  of  this  ANN  is  situational  awareness  in  combat 
fighter  tactics,  limited  to  a  2v2  visual  identification  intercept  training  flight.  Four  subnetworks  make 
up  this  ANN  model,  one  each  modeling  Ps  (probability  of  success),  /,  k,  and  w,  two  of  which  are  fed  by 
a  common  gated  dipole  field.  The  authors  proposed  that  Ps,  which  is  a  composite  of  probability  of  kill 
and  probability  of  survival,  is  critical  to  understanding  SA  and  workload  in  this  fighter  environment. 
Indeed,  in  the  air  combat  simulation  run,  the  Ps  measure  was  the  most  sensitive  to  changes  in  the 
tactical  situation,  and  it  correlated  highly  with  the  value  ofk.  Goodson  et  al.  describe  other  potential 
uses  for  an  SA  ANN.  Demonstration  softwaie  code  and  scenario  data  are  included  in  the  appendices. 

KEYWORDS:  air-to-air  mission,  aviation  (military),  combat  simulation,  modeling,  training 


86 


76.  Gordon,  B.  (1993).  Cockpit  attention  and  task  management:  Situation  awareness 
in  the  tactical  fighter.  In  Proceedings  of  the  Seventh  International  Symposium  on 
Aviation  Psychology  (Volume  2,  pp.  912-917).  Columbus:  The  Ohio  State 
University. 

The  article  describes  the  development  and  implementation  . of  a  cockpit  resource  management  (CRM) 
course  for  single  seat  tactical  aircraft  flown  by  the  United  States  Air  Forces  in  Europe  (USAFE).  The 
course  was  called  Cockpit  Attention  and  Task  Management  (CATM).  CATM  was  developed  as  a 
response  for  accident  investigations  that  suggested  that  every  serious  operational  mishap  involved  a 
deficiency  in  attention  management.  CATM  was  designed  to  teach  attention  management.  Attention 
management  was  defined  as  (p.  913),  "The  ability  to  prioritize  tasks,  recognize  mission  demands,  and 
maintain  situation  awareness  in  the  dynamic  flight  environment."  According  to  the  author,  attention 
management  became  synonymous  with  SA  within  the  program.  The  attention  management  training 
was  conducted  as  workshops  with  USAFE  pilots.  Although  the  author  reports  that  the  program  was 
well  received,  there  is  no  objective  data  to  support  its  effectiveness.  The  author  suggests  that  better 
metrics  for  measuring  the  effectiveness  of  such  human  factors  programs  are  needed. 

KEYWORDS:  attention,  aviation  (military),  cockpit  resource  management  (CRM),  cognitive 
processes,  definition,  pilot  error,  training 


77.  Granda,  T.M.,  McClure,  D.H.,  and  Fogarty,  J.W.  (1991).  The  development  of  an 
altitude  awareness  program:  An  integrated  approach.  In  Proceedings  of  the 
Human  Factors  Society  35th  Annual  Meeting  (Volume  1,  pp.  47-51).  Santa 
Monica,  CA:  Human  Factors  Society. 

The  research  was  not  directly  presented  as  a  SA  project,  but  the  concept  of  awareness  played  a  vital 
role  in  the  project.  A  rising  trend  of  altitude  deviations  had  been  detected  in  commercial  aviation.  This 
encouraged  the  development  of  the  Altitude  Awareness  Program  by  US  Air  and  the  Air  Line  Pilots 
Association.  The  paper  reviews  two  aspects  of  this  program:  proposed  cockpit  procedural  changes  to 
increase  crew  awareness  of  assigned  and  actual  altitude,  and  an  altitude  awareness  study  to  examine  the 
causes  of  altitude  deviations.  The  study  consisted  of  requesting  approximately  5500  USAir  pilots  to 
voluntarily  fill  out  a  questionnaire  developed  by  subject-matter  experts  to  identify  the  causes  and 
context  of  observed  altitude  deviations  in  actual  airline  operations.  As  of  the  writing  of  this  paper  a 
total  of  97  incident  reports  had  been  submitted.  Among  the  findings,  a  large  percentage  of  the  altitude 
deviations  involved  failure  to  follow  the  new  cockpit  procedures.  Typically,  this  involved  a  lack  of 
communication  among  cockpit  members.  The  paper  serves  as  an  example  of  research  within  an 
operational  setting,  the  importance  of  communication  among  crew  members  to  foster  shared  awareness 
of  the  aircraft  situation,  and  the  utility  of  retrospective  surveys  as  a  data  source. 

KEYWORDS:  aviation  (civilian),  measurement,  performance-based  measures, 

questionnaire/survey  data,  retrospective  measures,  subjective  measures, 
team  SA 
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78.  Gravelle,  M.D.  (1991).  Chief  scientist  report  special:  Technical  inquiry  report  on 
situation  awareness.  CSERIAC  Gateway,  2(2),  14. 

Gravelle  discusses  some  of  the  difficulties  encountered  in  trying  to  extend  the  SA  concept  from  its 
most  common  aviation  setting  to  the  domain  of  command,  control,  and  communication  (C3) 
environments.  The  analysis  suggested  that  the  two  main  problems  were  the  lack  of  a  common 
definition  and  the  lack  of  accepted  metric  techniques.  In  regard  to  the  definition  issue,  the  author 
suggests  (p.  14):  "Generally  speaking,  SA  describes  an  operator's  internal  representation  or  mental 
model  of  the  immediate  surroundings  and  a  certain  zone  of  interest."  In  regard  to  the  measurement 
issue,  the  author  expresses  concern  about  whether  metrics  that  have  primarily  been  designed  to  assess 
SA  in  an  air  combat  environment  will  be  appropriate  to  C3  settings.  In  particular,  the  author  argues 
that  the  SA  concept  will  need  to  include  awareness  of  the  other  team  members. 

KEYWORDS:  definition,  measurement,  team  SA 


79.  Griffin,  G.R.  (1988).  Predicting  air  combat  maneuve.nng  (ACM)  performance.  In 
AGARD-CP-458,  Human  Behavior  in  High  Stress  Situations  in  Aerospace 
Operations  (pp.  22-1  to  22-13).  Neuilly  Sur  Seine,  France:  Advisory  Group  for 
Aerospace  Research  &  Development.  (AD-A2 12884) 

This  paper  describes  two  studies  conducted  at  the  Fleet  Fighter  Air  Combat  Maneuvering  (ACM) 
Readiness  Program,  Naval  Air  Station  Oceana,  Virginia.  The  purpose  of  the  studies  was  to  evaluate 
the  prediction  of  ACM  performance  by  using  perceptual  motor  and  cognitive  multitask  tests  (Study  I), 
and  to  assess  the  correlation  of  Tactical  Aircrew  Combat  Training  System  (TACTS)  performance 
measures  and  overall  ACM  grades  (OAG)  assigned  by  VF-43  (adversary  squadron)  personnel  (Study 
11).  Study  I  determined  that  OAG  were  significantly  and  positively  related  to  offensive  maneuvering, 
situational  awareness  (SA),  and  mutual  support  measures.  In  addition,  dichotic  listening  test 
measures,  derived  under  multitask  conditions,  could  reliably  predict  ACM  performance  (n=18  F-4 
pilots).  Study  II  found  that  Fleet  Fighter  OAG  can  be  predicted  by  an  objective  kill  difference 
composite  score,  and  subjective  measures  of  situational  awareness,  mutual  support  and  energy 
management  (Subjects:  125  F-14  pilots). 

KEYWORDS:  a'*'  combat  maneuvering  (ACM),  air-to-air  mission,  attention,  aviation 
(military),  cognitive  processes,  individual  differences,  measurement, 
perform.ance-based  measures,  subjective  measures,  training 
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80.  Haines,  R.F.,  and  Flatau,  C.  (1992).  Night  fiying.  Blue  Ridge  Summit,  PA:  TAB 
Books. 

This  book  is  intended  to  aid  pilots  with  night  flying  skills.  SA  is  suggested  as  one  important 
component  of  safe  flying.  SA  is  defined  as  (p.  43),  "one's  ability  to  remain  aware  of  everything  that  is 
happening  at  the  same  time  and  to  integrate  that  sense  of  awareness  into  what  one  is  doing  at  the 
moment."  SA  is  seen  by  the  authors  to  be  the  result  of  both  genetic  and  experiential  influences.  In 
addition,  the  authors  suggest  that  some  general  exercises  might  increase  an  individual's  SA  capabilities. 
Five  categories  of  exercises  are  suggested.  (1)  vision  and  attention  scanning  exercise,  (2)  listening 
exercise,  (3)  muscular  exercise,  (4)  cognitive  exercise,  and  (5)  integration.  However,  no  data  is 
presented  to  support  the  effectiveness  of  these  exercises. 

KEYWORDS;  attention,  aviation  (civilian),  cognitive  processes,  definition,  training 


81.  Halski,  D.J.,  Landy,  R.J.,  and  Kocher,  J.A.  (1991,  April).  Integrated  control  and 
avionics  for  air  superiority:  A  knowledge-based  decision-aiding  system.  In 
AGARD-CP-474,  Knowledge-Based  System  Applications  for  Guidance  and  Control 
(pp.  53-1  -  53-11).  Neuilly  Sur  Seine,  France:  Advisory  Group  for  Aerospace 
Research  &  Development.  (AD-A235715) 

The  purpose  of  the  Integrated  Control  and  Avionics  for  Air  Superiority  (ICAAS)  program,  sponsored 
by  the  Wright  Research  and  Development  Center,  is  to  develop  a  real-time,  knowledge-based 
decision-aiding  system  for  air  combat.  ICAAS-developed  critical  technologies  will  focus  on  improving 
tactical  fighter  kill/survive  capabilities  when  outnumbered  during  air  combat.  Situation  awareness 
information  and  recommended  actions  will  aid  the  pilot  in  selecting  attack/defend  options.  The  majority 
of  the  paper  presents  in-depth  descriptions  of  the  key  ICAAS  system  functions  (and  improved 
capabilities);  Attack  Management  (Situation  Awareness),  Tactics  (Situation  Assessment),  Attack 
Guidance  (Positional  Advantage),  Defensive  Assets  Manager  (Threat  Evasion),  and  Performance 
Monitor  (Performance  Advantage). 

KEYWORDS;  air-to-air  mission,  aviation  (military),  avionics,  decision  aiding,  decision 
making,  pilot  aiding 
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82.  Hamilton,  W.L.  (1987).  Situation  awareness  metrics  program  (SAE  Technical  Paper 
Series  No.  871767).  Warrendale,  PA:  Society  of  Automotive  Engineers. 

The  author  describes  a  four-part  program  to  develop  SA  metrics  to  support  the  design  of  fighter 
cockpits.  The  importance  of  SA  is  emphasized  (p.  1),  "Situation  awareness  has  historically  been 
regarded  by  fighter  pilots  as  a  dominant  factor  in  air  combat."  The  author  cites  a  definition  with  is  said 
to  be  accepted  by  the  Air  Force  (p.  1):  "Situation  awareness  is  knowledge  of  current  and  near-term 
disposition  of  both  friendly  and  enemy  forces  within  a  volume  of  space."  The  author  suggests  that  the 
definition  is  somewhat  weak  because  it  does  not  offer  any  real  guidance  to  designers.  The  author 
proposes  that  SA  must  be  understood  within  the  context  of  the  flow  of  necessary  information  to  the 
pilot.  The  information  must  be  filtered  for  relevance  and  displayed  effectively  to  avoid  overloading  the 
pilot.  The  program  that  is  proposed  is  described  as  having  four  phases:  identify  information 
requirements,  develop  candidate  measures,  establish  methodology  for  analyzing  the  information 
filtering  process,  and  analyze  pilot  information  management  of  high  rates  of  information  flow.  The 
candidate  measures  of  SA  are  not  described  in  detail  but  appear  to  emphasize  performance-based 
"measures  of  merit."  The  SA  measurement  would  also  be  complemented  by  the  use  of  a  workload 
assessment  battery. 

KEYWORDS:  aviation  (military),  avionics,  definition,  display  design,  measurement, 
performance-based  measures,  workload 


83.  Hansman,  R.J.,  Hahn,  E,,  and  Midkiff,  A.  (1991).  Impact  of  data  link  on  flight 
crew  situational  awareness.  In  Proceedings  of  the  Sixth  International  Symposium 
on  Aviation  Psychology  (Volume  1,  p.  551).  Columbus:  The  Ohio  State 
University. 

This  paper  announces  the  concern  for  the  potential  loss  of  situational  awareness  as  a  result  of 
incorporating  data-linked  air  traffic  control  commands  into  the  cockpit  of  commercial  airliners.  The 
concern  is  that  the  change  will  remove  the  pilot's  access  to  "party-line  information"  that  is  currently 
available  from  listening  to  the  air  traffic  controller's  communications  with  other  aircraft.  A  survey  of 
current  commercial  airline  pilots  indicated  that  the  pilots  consider  such  party  line  information  to  be 
important.  Also,  preliminary  studies  suggest  the  pilots  are  more  likely  to  accept  "bad"  clearances  when 
using  the  data-link  system  than  when  using  the  traditional  radio-based  clearance  system. 

KEYWORDS:  a'’’  traffic  control,  avionics,  aviation  (civilian),  data  link,  measurement, 
performance-based  measures,  pilot  aiding,  questionnaire/survey  data, 
subjective  measures 


90 


84.  Hanson,  R.F.  (1988).  Situation  awareness  and  the  PVI  link.  In  Proceedings  of  the 
AIAA/IEEE  8th  Digital  Avionics  Systems  Conference  (pp.  193-195).  Washington, 
DC:  American  Institute  of  Aeronautics  and  Astronautics. 

The  author  contends  that  SA  is  vital  for  mission  success  in  a  tactical  fighter  and  that  "Good  Situation 
Awareness  and  good  Pilot-Vehicle  Interface  (PVI)  are  rapidly  becoming  synonymous"  (p.  193).  The 
author  suggests  a  research  program  for  developing  a  methodology  that  would  allow  the  SA  needs  of  a 
pilot  to  be  specified  and  then  tested  by  appropriate  performance-based  measures  of  merit.  The  author 
suggests  that  this  approach  is  essential  to  making  SA  a  viable  consideration  in  the  design  of  the  PVI. 

At  the  time  of  the  writing  the  research  was  in  the  planning  stages,  so  no  results  are  presented. 

KEYWORDS:  air-to-air  mission,  aviation  (military),  display  design,  measurement, 
performance-based  measures 


85.  Harbour,  J.L,,  and  Hill,  S.G.  (1990).  Using  HSYS  in  the  analysis  of  human-system 
interactions:  Examples  from  the  off-shore  petroleum  industry.  In  Proceedings  of 
the  Human  Factors  Society  34th  Annual  Meeting  (Volume  2,  pp.  1 1 90-11 94), 
Santa  Monica,  CA:  Human  Factors  Society. 

This  article  describes  using  a  computational  human  information  processing  model  to  analyze  28 
accidents  that  occurred  in  the  offshore  petroleum  industry.  The  model  is  called  the  Humans-System 
(HSYS)  model  and  it  characterizes  all  human  action  in  five  sequential  steps:  input  detection,  input 
understanding,  action  selection,  action  planning,  and  action  execution.  The  authors  interpret  some  of 
the  results  in  SA  terms  (p.  1191),  "Situational  awareness  involves  the  real-time  detection  and 
understanding  of  various  system  and  environmental  inputs.  In  HSYS  terminology,  situational 
awareness  includes  both  input  detection  and  input  understanding." 

KEYWORDS:  definition,  modeling,  retrospective  measures,  subjective  measures 


86.  Hardy,  S.M.  (1992,  January).  Air  Force  hits  the  intelligence  fastball:  Bringing 

real-time  intelligence  to  the  F-16  cockpit.  Journal  of  Electronic  Defense,  15(^),  29, 
31,  69. 

The  article  discusses  Project  Fastball.  Project  Fastball  is  an  effort  by  the  Air  Warfare  Center  to  bring 
real-time  intelligence  data  to  an  F-16  cockpit.  The  system  consists  of  three  parts:  (1)  air  and  ground 
based  sensors,  (2)  a  data  collection  facility  to  analyze  all  of  the  data  and  broadcast  it  to  the  F-16,  and 
(3)  an  on-board  computer  to  generate  the  display  on  the  HUD.  The  system  had  already  passed  tests  in 
the  simulator,  in-flight  tests  at  Eglin  Air  Force  Base's  test  range,  and  in-flight  tests  at  Nellis  Air  Force 
Base's  Green  Flag  exercise.  The  system  is  described  as  a  tool  for  improving  SA. 

KEYWORDS:  aviation  (military),  avionics,  data  fusion,  data  link,  flight  test,  pilot  aiding 
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87.  Hardy,  S.M.  (1993,  December).  Raising  the  cockpit's  iQ.  Journal  of  Electronic 
Defense,  16{\2),  51-56. 

This  article  reviews  several  systems  being  developed  under  the  umbrella  term  of  real-time  intelligence 
to  the  cockpit  (RTIC).  RTIC  includes  systems  that  not  only  down-link  information  into  the  cockpit, 
but  up-link  current  information  from  one  cockpit  to  distribute  to  others.  The  potential  benefits  of  such 
systems  are  interpreted  within  an  SA  framework.  For  example  the  author  concludes  (p.  56),  "Whether 
for  self-defense,  targeting,  combat  ED,  or  any  one  of  a  number  of  situational  awareness  aspects, 
real-time  intelligence  to  the  cockpit  promises  to  bring  pilots  and  weapons  specialists  a  clearer  picture  of 
the  environment  than  previously  possible." 

KEYWORDS:  aviation  (miiitary),  avionics,  data  link,  flight  test,  pilot  aiding,  target 
identification 


88.  Hart,  S.G.  (1989).  Overview  of  NASA  rotorcraft  human  factors  research.  In 
American  Helicopter  Society  45th  Annual  Forum  Proceedings  (pp.  441-453). 
Alexandria,  VA:  American  Helicopter  Society. 

This  article  is  a  high-level  review  of  numerous  research  programs  that  were  being  conducted  by 
the  Rotorcraft  Human  Factors  Research  Branch  at  NASA  -  Ames  Research  Center.  Situational 
Awareness  is  identified  as  one  of  four  major  program  elements  for  the  branch.  This  program  element  is 
described  as  investigating  near-term  and  long-term  enhancements  for  the  helicopter  cockpit  to  improve 
pilot  SA.  Among  the  technologies  discussed  are;  design  of  visual  display  systems,  helmet-mounted 
displays  (HMDs),  3-D  audio,  and  map  displays. 

KEYWORDS:  3-D  audio  displays,  display  design,  helicopter,  helmet-mounted  displays 
(HMDs),  pilot  aiding 
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89.  Hart,  S.G.,  and  Chappel,  S.L.  (1983),  Influence  of  pilot  workload  and  traffic 
information  on  pilot's  situation  awareness.  In  Nineteenth  Annual  Conference  on 
Manual  Control  (fip.  4-26).  Cambridge:  Massachusetts  Institute  of  Technology. 

This  paper  reports  the  results  of  a  study  of  the  effects  of  a  cockpit  display  of  traffic  information 
(CDTI)  on  SA  and  workload.  The  operation  of  multiple  aircraft  within  a  terminal  control  airspace  was 
simulated.  On  half  of  the  trials  all  aircraft  were  CDTI-equipped,  on  the  other  half  only  some  of  the 
aircraft  were  CDTI-equipped.  Workload  was  also  manipulated  to  produce  low  and  high  workload 
conditions.  Workload  was  measured  by  self-rating  device  involving  periodically  pressing  a  button  to 
indicate  the  current  load.  SA  was  measured  by  a  "stop-action”  procedure.  This  was  a  memory  probe 
procedure  that  involved  the  abiupt  stopping  of  a  simulation  with  a  blanking  of  the  displays.  Subjects 
were  asked  to  fill  out  a  form  describing  all  of  the  traffic  that  existed  at  the  time  of  the  stoppage. 

Results  indicated  that  the  availability  of  CDTI  increased  reported  workload,  but  did  not  improve  the 
completeness  or  accuracy  of  the  reported  traffic  information.  Despite  the  increase  in  workload  the 
pilots  favored  the  implementation  of  CDTI.  The  scenarios  that  inflicted  higher  workload  also  tended  to 
depress  S  A  scores.  Regardless  of  the  availability  of  CDTI  or  the  level  of  workload,  the  overall  amount 
of  information  reported  during  the  stop-action  SA  assessment  was  very  little.  This  led  to  the 
suggestion  by  the  authors  that  in  future  research  (p.  25),  "Remembered  information  should  be  but  one 
element  in  the  analysis.  Information  usage  and  strategies  selected  during  the  simulation  should  be 
studied  as  well,  to  obtain  a  complete  analysis  of  the  information  extracted  from  a  traffic  display  as  a 
function  of  the  tasks  a  pilot  is  performing." 

KEYWORDS;  a*''  traffic  control,  aviation  (civilian),  display  design,  measurement,  memory 
probe  measures,  performance-based  measures,  pilot  aiding,  workload 


90.  Hartman,  B.O.,  and  Secrist,  G.E.  (1991).  Situational  awareness  is  more  than 

exceptional  vision.  Aviation,  Space,  and  Environmental  Medicine,  62,  1084-1089. 

The  authors  emphasize  that  situation  awareness  derives  from  cognitive  processes  more  than  exceptional 
sensory  capabilities.  They  also  suggest  that  situation  awareness  enhancement  must  include  more  than 
simply  redesign  of  avionics  and  displays.  As  they  put  it  (p.  1084);  "We  have  chosen  a  different 
approach;  to  treat  situational  awareness  as  a  skill  generically  exercised  by  all  pilots  in  all  aircraft  at  all 
stages  of  their  cockpit  careers  and  to  select  and  to  train  for  situation  awareness."  As  a  conceptual 
background  for  their  approach,  the  authors  review  three  theoretical  domains  in  terms  of  their 
applicability  to  situation  awareness  in  pilots;  (1)  Automatic  processes  are  seen  as  a  way  to  circumvent 
working  memory  capacity  limitations.  (2)  Processing  near-threshold  information  allows  information 
outside  the  limits  of  conscious  awareness  to  influence  performance.  (3)  Skilled  memory  organizes  the 
long-term  store  of  information  for  adaptive  real-time  use.  The  authors  suggest  that  training  in  these 
domains  could  increase  pilot  situation  awareness.  They  have  started  a  research  program  to  examine  the 
utility  of  training  to  increase  near-thi  eshold  information  processing. 

KEYWORDS;  cognitive  processes,  review  paper,  training 


93 


91,  Harwood,  K.,  Barnett,  B.,  and  Wickens,  C.  (1988,  April).  Situational  awareness:  A 
conceptual  and  methodological  framework.  In  Proceedings  of  the  Psychology  in 
the  Department  of  Defense  Eleventh  Symposium  (Tech.  Report  No. 
USAFA-TR'88-1,  pp.  316-320).  Colorado  Springs,  CO:  US  Air  Force  Academy. 
(AD-A1 98723) 

Although  no  concise,  original  definition  of  situation  awareness  is  offered,  the  authors  provide 
considerable  discussion  of  relevant  definitional  issues.  This  discussion  is  presented  within  the 
framework  of  situation  awareness  being  information  about  the  Where,  What,  NVho,  and  When  of  a 
situation.  The  temporal  component  is  emphasized  in  the  discussion.  In  discussing  relevant 
methodologies  for  investigating  situational  awareness  the  authors  discuss  two  main  issues;  (1) 
Capturing  the  mental  model  used  to  perform  a  task,  and  (2)  Embedding  tasks  within  dynamic  scenarios. 

KEYWORDS:  cognitive  processes,  definition,  measurement,  mental  models 


92.  Henderson,  B.W.  (1989,  June  19).  Kaiser  improves  helmet-mounted  display  to 
boost  pilot's  kill  capability.  Aviation  Week  &  Space  Technology,  130(25),  119, 
121. 

The  article  discusses  the  Agile  Eye  Plus  helmet-mounted  display  system.  Agile  Eye  Plus  is  a  second 
generation  system  that  allows  a  greater  field-of-view,  displays  more  types  of  information,  and  has  some 
additional  convenience  features  than  the  first  generation  system  (Agile  Eye).  According  to  a 
representative  of  Kaiser  the  Agile  Eye  Plus  system  "increases  the  pilot's  situational  awareness"  along 
with  other  capabilities.  Simulator  and  in-flight  tests  have  demonstrated  dramatic  increases  in  the  kill 
ratio  for  pilots  using  the  Agile  Eye  Plus.  In  some  cases,  the  availability  of  the  system  more  than  offset 
experience  differences  among  participating  pilots.  Reportedly,  all  pilots  that  have  used  the  system  were 
positive  about  its  performance.  The  operational  capabilities  of  the  current  system  are  discussed,  and 
potential  future  developments  (including  the  integration  of  data  linked  information  onto  the  display)  are 
suggested. 

KEYWORDS:  aviation  (military),  combat  simulation,  data  fusion,  data  link,  display  c'esign, 
flight  test,  helmet-mounted  displays  (HMDs),  measurement, 
performance-based  measures,  pilot  aiding,  subjective  measures 


94 


93.  Henderson,  B.W.  (1991,  January  7).  Aircraft-sty ie  avionics  add  punch  to  U.S. 
Army's  next-generation  tank.  Aviation  Week  &  Space  Technology,  134(^),  45, 
47-48. 

The  article  discusses  proposed  "vetronics"  (a  contraction  of  "vehicle  electronics")  to  be  included  in  the 
M1A2  Abrams  main  battle  tank,  Among  other  things,  the  vetronics  include  an  integrated  command  and 
control  network,  inertial  navigation  and  fire  control  systems,  and  digital  displays.  This  package  will 
make  the  tank's  control  area  very  similar  to  current  fighter  aircraft  designs  and  is  expected  to  double 
defensive  battle  efficiency  and  increase  offensive  capability  by  50%.  Gordon  England,  vice  president  of 
engineering  and  research  at  General  Dynamics  Land  Systems  Division,  is  quoted  as  saying  (p.  47), 
"Using  the  laser  range  finder,  targets  can  be  precisely  located,  and  knowing  where  you  are,  you  can 
communicate  the  position  of  targets  to  other  tanks  in  the  force.  Situational  awareness  will  be  much 
better." 

KEYWORDS:  avionics,  battlefield  SA,  data  link 


94.  Henderson,  B.W.  (1992,  March  23).  NASA  Ames  pushes  automation  toward 
human-centered  design.  Aviation  Week  &  Space  Technology,  136(M),  69-70, 

The  article  describes  NAS  A- Ames'  research  with  "human-centered  automation."  The  goal  is  to 
implement  automation  while  helping  the  pilot  maintain  SA.  The  usual  approach  to  implementing 
automation  is  described  as  "management  by  exception."  In  a  management -by-exception  scheme  the 
automation  proceeds  with  actions  unless  the  human  crew  acts  to  stop  it.  The  human  crew  operates  as 
monitors  of  the  automation's  actions.  This  passive  monitoring  is  considered  to  be  incompatible  with 
maintaining  good  SA.  The  alternative  "human-centered"  approach  to  implementing  automation  would 
be  to  use  "management  by  consent."  In  a  management-by-consent  scheme,  pilot  action  is  required 
before  even  automatic  maneuvers  can  be  executed. 

KEYWORDS:  aviation  (civilian),  avionics,  pilot  aiding 


95.  Henderson,  B.W.  (1992,  November  23).  Automation  system  gains  acceptance. 
Aviation  Week  &  Space  Technology,  137{2^),  97-98. 

The  Center/Tracon  Automation  System  (CTAS)  is  a  collaborative  FAA/NASA  software  project  to 
provide  air  traffic  controllers  with  an  automated  decision  aid  to  increase  the  flow  of  aircraft  in  and 
around  airports.  It  consists  of  three  parts:  (1)  Traffic  Management  Advisor  -to  help  plan  the  most 
efficient  landing  order  and  assign  landing  times,  (2)  Final  Approach  and  Spacing  Tool  -  to  assist  in 
guiding  aircraft  to  the  appropriate  runway,  and  (3)  Descent  Advisor  -  to  calculate  cruise  and  descent 
clearances.  The  Traffic  Management  Advisor  was  tested  in  Denver  and  managers  there  reported  that  it 
helped  their  situational  awareness.  The  primary  benefits  of  the  system  are  seen  as  a  reduction  in  the  air 
traffic  controllers'  workload  combined  with  an  increase  in  SA. 

KEYWORDS:  air  traffic  control,  decision  aiding,  decision  making,  workload 


95 


96.  Henderson,  E.D.  (1988).  Air  traffic  controiier  awareness  and  resource  training 
(SAE  Technical  Paper  Series  No.  881518).  Warrendale,  PA:  Society  of 
Automotive  Engineers. 

Human  error  is  cited  as  the  cause  of  a  very  large  proportion  (over  90%)  of  the  operational  errors 
observed  in  the  air  traffic  control  system.  A  major  contributor  to  these  errors  may  be  a  lack  of  team 
skills  and  communication  among  air  traffic  controllers.  The  article  describes  the  Controller  Awareness 
and  Resource  Training  (CART)  program.  CART  is  an  attempt  to  provide  training  analogous  to  airline 
Cockpit  Resource  IVfanagement  (CRM)  training  to  air  traffic  controllers.  Of  the  fourteen  sessions  or 
lessons  that  constitute  the  CART  training,  one  lesson  is  devoted  to  SA.  More  specifically,  the  lesson 
discusses  (p.  3)  the  "ability  to  recognize  the  elements  of  situational  awareness  when  it  may  have  been 
lost,  and  how  to  regain  or  maintain  it."  The  goal  of  the  CART  program  is  to  improve  the  safety  and 
efficiency  of  the  air  traffic  system. 

KEYWORDS:  air  traffic  control,  cockpit  resource  management  (CRM),  team  SA,  training 


97.  Hitchcock,  L.  (1994).  Ycurs,  .mine,  and  ours:  Some  observations  on  the 
metaphysics  of  situational  awareness.  In  R.D.  Gilson,  D.J.  Garland,  and  J.M. 
Koonce  (Eds,),  Situational  awareness  in  compiex  systems:  Proceedings  of  a 
CAHFA  Conference  (pp.  3-16).  Daytona  Beach,  FL:  Embry-Riddle  Aeronautical 
University  Press. 

The  author  attempts  to  address  the  potential  effects  of  an  individual  observer  on  SA,  especially  the 
effect  on  communicating  SA  from  one  person  to  another  person  (such  as  in  air  traffic  control).  To 
attack  this  issue  Hitchcock  uses  the  "pseudo"  pyramid  representation  of  SA  developed  by  Harwood, 
Barnett,  and  Wickens  (1988)  to  represent  the  where,  what,  when,  and  who  components  of  SA. 
Hitchcock  contends  that  each  individuals  unique  approach  to  SA  can  be  represented  by  a  "Cognitive 
Centroid"  that  is  located  somewhere  within  the  conceptual  space  of  the  "pseudo"  pyramid.  The  quality 
of  communicated  SA  will  be  influenced  by  the  distance  between  the  locations  of  the  cognitive  centroids 
of  .  le  individuals  involved. 

KEYWORDS:  cognitive  processes,  individual  differences,  team  SA 


98.  Hoover,  G.W.  (1992,  June  15).  Stop  saturating  pilots  with  alphanumeric 
hieroglyphics.  Aviation  Week  &  Space  Technoiogy,  736(24),  98-99. 

The  article  argues  that  there  is  an  excessive  reliance  on  presenting  pilots  obscure  alphanumeric 
information.  The  author  argues  for  pictorial  displays  that  would  simulate  the  cues  of  the  natural  world. 
As  ’  "irt  of  the  justification  for  this,  the  author  states  (p.  99),  "Research  has  proved  that  good  pictorial 
visu  I  information  is  a  powerful  counter  to  vertigo  and  will  rapidly  restore  a  pilot's  situational 
awi  mess.  It  should.  He  is  using  the  same  cues  in  flight  that  he  has  used  every  day  since  he  first 
ope  3d  his  eyes." 

KE  'WORDS:  aviation  (civilian)  display  design,  pilot  aiding 
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99.  Hopkin,  V.D.  (1994),  Situational  awareness  in  air  traffic  control.  In  R.D.  Gilson, 
D.J.  Garland,  and  J.M.  Koonce  (Eds.),  Situational  awareness  in  complex  systems: 
Proceedings  of  a  CAHFA  Conference  (pp,  171-178).  Daytona  Beach,  FL: 
Embry-Riddle  Aeronautical  University  Press. 

The  author  reviews  some  of  the  conceptual  issues  involved  in  applying  the  S  A  concept  to  air  traffic 
control.  In  discussing  S  A  in  air  traffic  control  the  author  connects  it  to  the  concept  of  the  controller's 
"picture"  (p.  174);  "In  air  traffic  control,  situational  awareness  seems  to  correspond  quite  well  with  the 
concept  of  the  controller's  picture.  This  picture  is  sometimes  construed  as  an  example  of  a  mental 
model  but  although  the  picture  includes  the  controller's  mental  model  it  is  not  confined  to  it,  being  a 
more  dynamic  entity  than  most  mental  models  in  that  it  incorporates  changing  states  and  their 
consequences."  The  potential  effects  of  individual  differences,  training,  and  automation  are  also 
discussed.  In  particular,  many  of  the  current  plans  to  automate  the  air  traffic  controller's  job  are  seen  as 
possibly  degrading  the  SA  achievable  by  future  air  traffic  controllers. 

KEYWORDS:  aT  traffic  control,  cognitive  processes,  individual  differences,  mental 
models,  training 


100.  Hoshstrasser,  B.D.,  and  Small,  R.L.  (1994).  The  impact  of  associate  systems 

technology  on  an  air  traffic  controller's  situational  awareness.  In  R.D.  Gilson,  D.J. 
Garland,  and  J.M.  Koonce  (Eds.),  Situational  awareness  in  complex  systems: 
Proceedings  of  a  CAHFA  Conference  (pp.  227-236).  Daytona  Beach,  FL: 
Embry-Riddle  Aeronautical  University  Press. 

The  authors  discuss  the  issues  associated  with  applying  artificially-intelligent  associate  systems  to 
aiding  air  traffic  controllers.  They  identify  numerous  potential  payoffs  and  pitfalls,  and  make 
suggestions  for  future  applications. 

KEYWORDS:  air  traffic  control,  design  guidelines 


101.  Houck,  R.,  Kelly,  B.D.,  and  Wiedemann,  J.  (1986).  An  Integrated  display  for 
vertical  situation  awareness  in  commercial  transport  aircraft  (SAE  Paper  No. 
861770).  Warrendale,  PA:  Society  of  Automotive  Engineers. 

One  of  the  most  frequently  reported  safety  violations  in  commercial  aviation  is  deviation  from  assigned 
altitude.  The  authors  suggest  that  this  is  because  there  is  no  adequate  display  of  the  aircraft's  vertical 
situation.  They  argue  that  a  well-designed  vertical  situation  display  (VSD)  would  provide  a  significant 
benefit  to  the  commercial  aircraft  cockpit.  As  they  put  it  (p.  61):  "With  the  advent  of  new  CRT 
technology,  many  innovative  pictorial  display  presentations  are  possible  to  depict  the  airplane's  position 
in  space.  These  pictorial  displays  incorporate  features  that  take  advantage  of  the  pilot's 
decision-making  and  discriminatory  capabilities,  thereby  resulting  in  greater  situational  awareness, 
better  planning,  and  reduced  workload,"  Alternative  approaches  to  the  design  of  such  a  display  are 
described  and  contrasted.  However,  at  the  time  of  the  writing,  no  empirical  evaluation  of  any  of  the 
display  formats  had  been  conducted. 

KEYWORDS;  aviation  (civilian),  display  design,  pilot  aiding,  pilot  error,  workload 
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102.  Houck,  R.D.,  Rogers,  W.H.,  and  Braune,  R.J.  (1987).  Advanced  technology 
cockpit  design  and  the  management  of  human  error  (SAE  Paper  No.  872525). 
Warrendale,  PA:  Society  of  Automotive  Engineers. 

In  addressing  the  issue  of  pilot  error  in  commercial  aviation  the  authors  suggest  that  advanced 
technology  in  the  cockpit  must  be  carefully  employed.  As  one  aspect  of  this  discussion,  they 
emphasize  that  the  use  of  advanced  technology  (such  as  automation  or  pictorial  displays)  should  be 
guided  by  the  need  to  increase  pilot  situational  awareness.  SA  is  thus  seen  as  beneficial  to  the 
reduction  of  pilot  error.  The  authors  also  suggest  that  new  control  laws  in  the  cockpit  could  reduce 
manual  control  workload,  which  could  allow  more  time  for  maintaining  better  SA. 

KEYWORDS:  aviation  (civilian),  avionics,  display  design,  pilot  aiding,  pilot  error, 
workload 


103.  Howell,  W.C.  (1393).  Engineering  psychology  in  a  changing  world.  Annual 
Review  of  Psychology,  44,  231-263. 

As  part  of  a  review  of  the  larger  topic  of  engineering  psychology,  the  author  devotes  a  few  pages  to 
discussing  the  status  of  the  SA  concept  and  SA  research.  Addressing  the  definitional  issue,  the  author 
states  (p.  241),  "While  definitions  vary,  the  essential  idea  is  that  an  operator  must  keep  track  of  a  lot  of 
information  from  a  variety  of  sources  over  time  and  organize  or  interpret  this  information  to  behave 
appropriately."  Numerous  studies  are  discussed,  but  few  real  conclusions  could  be  reached  from  the 
existing  data.  The  main  points  made  are.  (1)  the  conceptual  underpinnings  must  be  identified  and 
explored,  (2)  the  pressure  to  produce  a  useful  measure  and  immediately  apply  it  to  selection,  training,  or 
design  is  growing  and  could  be  dangerous,  and  (3)  it  would  be  a  mistake  to  allow  any  existing  measure 
of  SA  to  prematurely  become  the  "accepted  operationalization  of  SA." 

KEYWORDS:  cognitive  processes,  definition,  implicit  measures,  measurement,  memory 
probe  measures,  mental  models,  review  paper,  SAGAT,  SART,  subjective 
measures,  workload 


104.  Hubbard,  D.C.,  Rockway,  M.R.,  and  Waag,  W.L.  (1989).  Aircrew  performance 
assessment.  In  R.S.  Jensen  {Ed.),  Aviation  psychology  (pp.  342-377).  Aldershot, 
United  Kingdom:  Gower. 

The  authors  describe  the  need  for  good  performance  measures  of  pilots  or  aircrews.  These  measures 
would  be  useful  for  aircrew  selection,  training,  and  research  and  development.  Many  approaches  are 
described  and  discussed.  Within  this  context  the  authors  express  reservations  about  the  concept  of  SA 
(p.  375),  "Within  the  tactical  domain,  the  concept  of  situational  awareness  is  used  quite  frequently  to 
describe  skills  in  maintaining  an  awareness  of  the  tactical  situation.  For  multicrew  aircraft,  the  concept 
of 'captaincy'  is  often  used  to  describe  skills  unique  to  assuming  the  role  of  aircraft  commander. 
Unfortunately,  there  are  no  universally  accepted  definitions  of  these  concepts.  It  would  not  be 
uncommon  to  have  five  pilots  give  five  different  definitions  for  such  concepts.  It  is  apparent  that 
attempts  to  develop  measures  of  such  ill-defined  concepts  are,  at  best,  extremely  difficult." 

KEYWORDS:  aviation  (military),  definition,  measurement,  performance-based  measures 
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105.  Hughes,  0.  (1992,  March  9).  JTIDS  links  air,  land,  sea  forces  in  first  tri-service 
cooperative  tests.  Aviation  Week  &  Space  Technoiogy,  136(^0),  56-57. 

The  Joint  Tactical  Information  Distribution  System  (JTIDS)  was  successfully  tested  as  a  tool  to 
integrate  tactical  information  among  units  from  the  Air  Force,  Army,  and  Navy.  JTIDS  data  links  were 
successful  in  putting  six  military  aircraft,  a  Navy  ship,  two  ship  simulators,  and  two  Army  ground 
terminals  together  in  a  network.  Within  this  network,  sensor  data  could  be  exchanged  in  real-time. 

This  permitted  new  air-to-air  tactics  that  would  not  be  possible  without  the  data  link.  Summing  up  the 
potential  value  of  the  system,  the  author  states  (p.  57),  "Increased  situational  awareness  will  make  it 
possible  to  respond  faster  to  a  threat  with  a  coordinated  offensive  or  defensive  move." 

KEYWORDS:  air-to-air  mission,  aviation  (military),  avionics,  battlefield  SA,  data  link,  flight 
test,  naval  SA,  pilot  aiding 


106.  Hughes,  D.  (1992,  October  12).  Cameras  could  give  pilots  exterior  views  of 
aircraft.  Aviation  Week  &  Space  Technoiogy,  /57(15),  32-33. 

The  addition  of  external  cameras  to  British  Airways  Boeing  747s  is  discussed  as  an  accident  prevention 
and  problem  diagnosis  tool.  The  cameras  would  allow  the  flight  crew  to  visually  inspect  the  aircraft's 
engines  from  the  cockpit,  even  in  flight.  Used  in  this  way,  the  cameras  are  described  as  "situational 
awareness  aids"  (p.  32). 

KEYWORDS;  aviation  (civilian),  pilot  aiding 


107.  Hughes,  D.  (1992,  November  23).  Boeing,  United  pursue  major  cockpit  advance. 
Aviation  Week  &  Space  Technoiogy,  757(21),  89-91. 

Boeing  and  United  Airlines  are  working  together  to  develop  the  Enhanced  Situation  Awareness  System 
(ESAS)  for  commercial  aircraft.  The  system  will  fuse  data  from  a  variety  of  sensor  systems  (such  as 
forward  looking  infrared  cameras,  millimeter-wave  and  x-band  radar,  video  cameras,  etc.)  that  could 
improve  terrain  and  obstacle  avoidance,  landings,  and  flight  through  hazardous  weather  conditions. 

The  goal  is  to  "place  the  pilot  in  a  VFR-like  situation  at  all  times"  (p.  89).  It  is  expected  that  this 
system  would  increase  safety,  decrease  delays  due  to  weather,  and  allow  aircraft  more  access  to  airports 
with  limited  navigation  and  landing  aids,  The  greatest  technical  challenge  for  the  system  designers  is 
expected  to  be  the  human-machine  interface, 

KEYWORDS:  aviation  (civilian),  avionics,  data  fusion,  display  design,  pilot  aiding 


99 


108.  Hughes,  D.  (1993,  July  26).  USAF,  Navy  pursue  NVGs  for  fighters,  bombers. 
Aviation  Week  &  Space  Technoiogy,  139[A],  44-45. 

The  Air  Force  and  Navy  are  developing  new  night-vision  goggle  systems  designed  specifically  for 
fixed-wing  aircraft  (fighters,  bombers,  transport,  and  tankers).  The  logic  for  this  program  was 
described  as  follows  (p.  44),  "The  systems  can  be  used  for  low-level  navigation  at  night,  night 
refiieling,  landing  at  blacked-out  airfields  and  even  identification  and  tracking  of  an  intruder  aircraft  by 
an  interceptor.  The  systems  are  intended  to  enhance  situational  awareness  rather  than  to  aid  the  pilot  in 
air-to-air  or  air-to-ground  target  acquisition." 

KEYWORDS;  aviation  (military),  helmet-mounted  displays  (HMDs),  pilot  aiding,  target 
identification 


109.  Hughes,  D.,  and  Lenorovitz,  J.M.  (1991,  June  24).  F-117A's  performance  boosts 
wide  range  of  improvements.  Aviation  Week  &  Space  Technoiogy,  734(25), 
20-23. 

The  article  describes  planned  upgrades  to  the  Air  Force's  F-1 17A  Stealth  fighter.  Among  the  changes 
proposed  for  the  F-1 17A  are  additions  to  the  cockpit  (p.  21),  "The  F-1 17's  offensive  capability  also  is 
being  improved  through  the  addition  of  Honeywell  color  multi-function  displays  with  the  capability  to 
integrate  a  digital  moving  map  by  Harris  Corp.  to  give  pilots  greater  situational  awareness."  It  was 
believed  that  this  program  marked  the  first  operational  use  of  military-qualified  digital  map  technology 
in  a  fighter-categoty  aircraft.  It  was  also  hoped  that  the  cockpit  changes  would  reduce  the  potential  for 
accidents  caused  by  pilot  disorientation. 

KEYWORDS:  aviation  (military),  data  fusion,  display  design,  pilot  aiding 
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110.  Hughes,  E.R.,  Hassoun,  J.A.,  Ward,  G.F.,  and  Rueb,  J.D.  (1990,  July).  An 

assessment  of  selected  workload  and  situation  awareness  metrics  in  a  part-mission 
simulation  (Tech.  Report  No.  ASD-TR-90-5009).  Wright-Patterson  Air  Force  Base, 
OH:  Aeronautical  Systems  Division. 

This  experiment  investigated  the  use  of  SAGAT  and  SA-SWOKD  as  situation  awareness  metrics  in 
air-to-air  part  mission  simulator.  Situation  awareness  was  manipulated  by  sensor  and  display  type. 

Half  of  the  trials  used  a  typical  forward-looking  radar  displayed  on  a  current  fire-control  radar  (FCR) 
display.  The  other  half  of  the  trials  simulated  the  use  of  data-linked  information  from  off-board  sensors 
along  with  radar  information  displayed  on  a  Horizontal  Situation  Format  (HSF)  display.  SAGAT  was 
insensitive  to  the  display  configuration  manipulation.  However,  there  was  concern  that  the  small 
number  of  trials,  limited  training  time  for  subjects,  and  experimental  design  might  not  have  been 
compatible  with  a  SAGAT  analysis.  SA-SWORD  was  sensitive  to  the  display  configuration 
manipulation.  The  pilots  felt  that  the  HSF  display  significantly  improved  situation  awareness.  The 
study  also  included  analysis  of  performance  measures  and  subjective  and  physiological  workload 
measures. 

KEYWORDS:  aviation  (military),  combat  simulation,  data  fusion,  data  link,  display  design, 
measurement,  memory  probe  measures,  performance-based  measures,  pilot 
aiding,  SAGAT,  SA-SWORD,  subjective  measures,  workload 


Ul.  Johnston,  J.C.,  Horlitz,  K.L.,  and  Edmiston,  R.W.  (1993).  Improving  situation 
awareness  displays  for  air  traffic  controllers.  In  Proceedings  of  the  Seventh 
International  Symposium  on  Aviation  Psychology  (Volume  1,  pp.  328-334). 
Columbus:  The  Ohio  Slate  University. 

This  paper  reviews  the  results  of  four  laboratory  visual  search  experiments  that  compared  different 
display  approaches  for  air  traffic  controller's  Situation  Awareness  Displays  (SADs).  The  goal  was  to 
use  display  formatting  that  emphasized  the  natural  perception  of  the  important  information  rather  than 
the  cognitive  calculation  of  a  display's  meaning.  As  a  test  case,  the  authors  examined  different 
approaches  for  representing  the  altitudes  of  aircraft  on  the  controller's  SAD  display  The  dependent 
measure  was  generally  the  response  time  for  correct  responses.  The  results  found  that  coding  the 
aircrafts'  altitudes  by  color  was  most  effective,  using  achromatic  coding  schemes  (such  as  gray-scale  or 
dot-striping)  showed  some  promise,  and  coding  based  on  size  or  shape  were  generally  ineffective.  In 
addition  to  these  specific  findings,  the  authors  argue  that  the  results  support  sma'l  laboratory  part-task 
evaluations  as  a  viable  way  to  "sort  out  promising  from  unpromising  display  schemes"  (p.  332). 

KEYWORDS:  S'r  traffic  control,  display  design,  measurement,  performance-based 
measures 


101 


112,  Judge,  C.L.,  and  Bushman,  J.B.  (Panel  Co-Chairs)  (1992).  Situation  awareness: 
Modeling,  measurement,  and  impacts.  In  Proceedings  of  the  Human  Factors 
Society  36th  Annual  Meeting  (Volume  1,  pp.  40-42).  Santa  Monica,  CA:  Human 
Factors  Society. 

This  set  of  abstracts  summarizes  a  panel  session.  The  Introduction  presents  a  definition  of  SA  that  has 
been  accepted  by  the  Air  Force  operational  community  (p.  40),  "A  pilot's  continuous  perception  of  self 
and  aircraft  in  r  elation  to  the  dynamic  environment  of  flight,  threats,  and  mission  and  the  ability  to 
forecast,  then  execute  tasks  based  on  that  perception,"  The  goal  of  the  panel  was  to  present  current 
research  in  the  role  of  SA  in  aviation  and  speculate  on  future  directions.  Five  abstracts  follow  the 
introduction:  Michael  Vidulicb  {Measuring  Situation  Awareness),  Michael  Houck  {Situation 
Awareness  in  Beyond-Visual-Range  Air  Combat),  Frederick  M.  Seim  and  David  Perry  {A  Test  Battery 
to  Predict  Individual  Differences  in  Situational  Awareness),  J.  Raymond  Comstock 
{Human/Automation,  Integration,  and  Situation  Awareness),  and  Cynthia  Dominguez  {Visual 
Identification  in  Near-Threshold  Processing). 

KEYWORDS:  aviation  (civilian),  aviation  (military),  cognitive  processes,  definition, 
individual  differences,  measurement,  pilot  aiding,  training 


113.  Kaempf,  G.L.,  Wolf,  S.,  and  Miller,  T.E.  (1993).  Decision  making  in  the  AEGIS 
Combat  Information  Center.  In  Proceedings  of  the  Human  Factors  and 
Ergonomics  Society  37th  Annuai  Meeting  (Volume  2,  pp.  1 1 07-1  111).  Santa 
Monica,  CA:  Human  Factors  and  Ergonomics  Society. 

The  paper  reviews  an  evaluation  of  the  decision  processes  of  anti-air  warfare  (AAW)  officers  in  the 
Combat  Information  Center  (CIC)  of  an  AEGIS  cruiser.  AEGIS  cruisers  combine  advanced  radar 
systems  with  anti-  aircraft  missiles  to  provide  air  defense  for  Navy  task  forces.  Interview  data  was 
collected  from  28  experienced  AAW  officers  discussing  14  separate  operational  incidents.  The 
procedures  followed  in  setting  up  the  interviews,  extracting  the  knowledge,  and  creating  the 
recommendations  were  inspired  by  Klein's  Naturalistic  Decision  Making  (NDM)  research.  Data 
regarding  183  operational  decisions  were  collected.  The  selection  of  a  course  of  action  in  these 
incidents  was  typically  (i.e.,  95%  of  the  time)  determined  by  the  formation  of  a  situation  assessment  by 
the  operator.  Only  in  5%  of  the  cases  were  multiple  options  generated  and  compared.  Given  that  the 
situation  assessment  discussed  by  NDM  researchers  can  be  considered  one  form  of  situation 
awareness,  this  result  suggests  that  SA  is  a  vital  component  of  real-world  performance.  Approaches 
for  designing  decision  aids  are  also  discussed. 

KEYWORDS:  cognitive  processes,  decision  aiding,  decision  making,  naval  SA, 

questionnaire/survey  data,  retrospective  measures,  subjective  measures 
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114,  Kass,  S.J.,  Herschler,  D.A.,  and  Companion,  M.A.  (1990).  Are  they  shooting  at 
me?:  An  approach  to  training  situational  awareness.  In  Proceedings  of  the  Human 
Factors  Society  34th  Annual  Meeting  (Volume  2,  pp.  1352-1356).  Santa  Monica, 
CA:  Human  Factors  Society. 

This  paper  describes  a  study  in  which  the  authors  investigated  a  training  strategy  to  improve  situational 
awareness.  Twenty  civilians  (University  of  Central  Florida  volunteers),  divided  into  two  groups,  were 
trained  in  location  identification  on  a  tank  muzzle  flash  task.  One  group  received  only  flashes  in 
training;  members  of  the  second  group  were  presented  with  flashes  plus  additional  stimuli  as  clutter. 
During  testing  (random  flashes,  with  clutter),  subjects  in  the  first  group  scored  significantly  better. 
These  results,  according  to  the  authors,  suggest  that  "pattern  recognition  skills  are  best  developed  in  a 
minimum-stimulus  condition."  The  authors  also  incorporate  brief  discussions  of  Rasmussen's 
progressive  levels  of  cognitive  processing  (knowledge-based,  rule-based,  and  skill-based  behaviors), 
Teichner's  task  theory,  signal  detection  theory,  and  state-dependent  learning  theory  into  the  explanation 
of  their  study. 

KEYWORDS:  battlefield  SA.  cognitive  processes,  measurement,  performance-based 
measures,  training 


115,  Kass,  S.J.,  Herschler,  D.A.,  and  Companion,  M.A.  (1991).  Training  situational 
awareness  through  pattern  recognition  in  a  battlefield  environment.  Military 
Psychology,  3,  105-112. 

This  paper  is  a  revised  and  slightly  expanded  version  of  the  Kass.  Herschler,  and  Companion  (1990) 
paper  and  is  reviewed  under  that  citation. 

KEYWORDS:  battlefield  SA,  cognitive  processes,  measurement,  performance-based 
measures,  training 
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116,  Kelly,  M.J.  (1988).  Performance  measurement  during  simulated  air-to-air  combat. 
Human  Factors,  30,  495-506. 

This  article  reports  on  different  approaches  for  automating  the  air  combat  maneuvering  (ACM) 
performance  assessment  of  pilots  flying  complex  air  combat  missions  in  high  fidelity  simulators. 
Problems  with  maintaining  SA  are  considered  part  of  the  justification  for  needing  such  systems  (p. 

496):  "An  ACM  performance  measurement  system  (ACMPMS)  can  provide  information  storage  and 
processing  that  are  complimentary  to  the  cognitive  abilities  and  limitations  of  the  human  pilot  and 
instructor  by  augmenting  their  memories  and  situational  awareness,  which  may  become  overloaded 
during  the  mission."  Describing  what  is  meant  by  the  term  "situation  awareness,"  the  author  states  (p. 
499);  "The  pilot  needs  to  be  keenly  aware  of  the  relative  positions  of  the  terrain  and  all  friendly  and 
adversary  forces  as  well  as  able  to  project  such  positions  and  actions  into  the  future.  Situation 
awareness  is  probably  the  sum  of  numerous  perceptual  and  cognitive  skills."  The  author  concludes  that 
an  algorithm  must  combine  information  about  the  superiority/inferiority  of  the  pilot's  aircrafr  relative  to 
the  adversary,  along  with  measures  of  control  use,  energy  management,  SA  and  team  support  to  be 
effective. 

KEYWORDS:  9'''  combat  maneuvering  (ACM),  air-to-air  mission,  aviation  (military), 

combat  simulation,  definition,  measurement,  performance-based  measures, 
subjective  measures,  team  SA,  training 


117,  Kibbe,  M.P.  (1988).  Information  transfer  from  intelligent  EW  displays.  In 

Proceedings  of  the  Human  Factors  Society  32nd  Annual  Meeting  (Volume  1,  pp. 
107-110).  Santa  Monica,  CA:  Human  Factors  Society. 

The  results  of  two  experiments  to  examine  the  effect  of  automation  in  the  combat  aircraft  cockpit  on 
situation  awareness  were  reported,  and  the  design  of  a  third  experiment  was  outlined.  The  motivation 
for  the  research  was  concern  that  automation  might  reduce  situation  awareness.  The  author  defined  SA 
as,  "The  particular  kind  of  SA  investigated  here  is  tactical  situational  awareness  (TSA)  which  includes 
kno'  /ledge  of  the  status  of  one's  own  aircraft,  other  aircraft,  and  ground  threats  and  targets"  (p.  107). 
The  effects  of  automation  on  SA  were  studied  by  having  the  subjects  perform  a  threat  recognition  task 
on  a  laboratory  simulation  of  a  Radar  Warning  Receiver  (RWR).  The  threat  recognition  task  required 
manual  inputs  in  Experiment  1,  but  was  automated  in  Experiment  2  and  only  required  monitoring.  The 
threat  recognition  task  was  performed  as  either  a  single  task  or  as  a  dual  task  combined  with  a  tracking 
task.  Although  not  identified  as  such  by  the  author,  the  performance  on  the  tracking  task  was  treated 
as  a  workload  metric.  Tracking  performance  was  degraded  by  the  addition  of  the  threat  recognition 
task,  even  in  the  automated  condition.  Tracking  performance  on  the  automated  dual  task  trials  was 
approximately  half-way  in-between  single  task  tracking  and  the  manual  threat  identification  dual  task 
condition  tracking  performance.  SA  was  measured  by  a  retrospective  questionnaire  filled  out  after  the 
trial  The  subjects  were  asked  to  reconstruct  the  timeline  of  threat  appearances  during  the  trial.  No 
effect  of  automation  on  the  accuracy  of  these  reconstructions  was  detected. 

KE  WORDS:  aviation  (military),  avionics,  definition,  measurement,  performance-based 
measures,  pilot  aiding,  questionnaire/survey  data,  retrospective  measures, 
subjective  measures,  target  identification,  workload 
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118.  King,  P.A.  (1987).  Big  picture:  A  solution  to  the  problem  of  situation  awareness. 
In  Proceedings  of  the  IEEE  1987  National  Aerospace  and  Electronics  Conference  - 
NAECON  1987  (Volume  1,  pp.  261-265).  New  York:  Institute  of  Electrical  and 
Electronics  Engineers. 

The  author  contends  that  one  of  the  major  difficulties  facing  the  pilot  of  contemporary  fighter  aircraft  is 
the  proliferation  of  displays.  As  aircraft  sensor  capabilities  have  increased  new  displays  have  been 
added  in  a  piece-meal  fashion  to  display  the  information  of  each  sensor  system  independently.  The 
pilot  must  then  fuse  the  information  from  these  different  displays.  The  time  available  is  often 
insufficient  for  the  information  to  be  properly  understood  by  the  pilot.  This  paper  describes  a  display 
concept  developed  by  McAir  and  the  Air  Force  Wright  Laboratory  to  address  this  problem.  The  display 
concept  is  called  "Big  Picture,"  and  involves  integrating  all  of  the  information  together  on  a  large 
(about  300  square  inch)  display  area.  A  variety  of  data  (e  g.,  ownship  position,  location  and  identity  of 
other  aircraft,  threat  and  target  locations,  route-of-flight,  etc.)  is  overlaid  on  a  background  display 
(either  a  digital  map  or  a  perspective  view  of  forward  terrain).  A  simulation  facility  to  test  this  concept 
is  described  and  some  technical  challenges  for  implementing  the  system  in  a  real  cockpit  are  discussed. 
The  author  states  (p.  265),  "Preliminary  data  indicate  that  the  Big  Picture  concept  improves  pilot 
situation  awareness  and  is  the  best  way  to  provide  the  data  collected  by  multiple  sensors." 

KEYWORDS;  aviation  (military),  big  picture  displays,  display  design,  pilot  aiding 


119.  Kinsley,  S.A.,  Warner,  N.W.,  and  Gleisner,  D.P.  (1985,  December).  A 

comparison  of  two  pitch  ladder  formats  and  an  ADI  ball  for  recovery  from  unusual 
attitudes  (Tech.  Report  No.  NADC-86012-60).  Warminster,  PA:  Naval  Air 
Development  Center.  (AD-A181253) 

This  study  was  initiated  in  part  due  to  suspicion  that  the  F/A-18  heads-up  display  (HUD)  symbology 
provides  inadequate  attitude  awareness.  Pilots  rated  this  HUD  "marginally  acceptable"  when  recovery 
from  unusual  attitudes  was  involved.  Two  experiments  were  performed.  One  used  static  depictions  of 
attitude  indicator  displays  showing  unusual  attitudes;  measures  were  decision  time  (time  to  initiate  a 
control  response)  and  error  rate  in  control  response.  The  current  HUD  pitch  ladder  was  compared  with 
an  improved  modification  of  the  HUD  pitch  ladder  symbology  and  with  an  ADI  ball.  Decision  time  for 
the  ADI  ball  was  significantly  faster  than  for  the  two  pitch  ladders.  The  second  experiment  evaluated 
the  current  HUD  display  and  the  ADI  ball  formats  in  a  dynamic  simulation.  Decision  time  and 
recovery  time  were  measured.  Decision  time  was  significantly  faster  against  a  clear  blue  sky  than 
against  a  night  sky  with  stars.  These  two  backgrounds  were  evaluated  because  of  the  strong 
conviction  by  the  authors  that  "the  lack  of  a  real,  visible  horizon  is  one  of  the  primary  causes  of  the  loss 
of  SA"  (p,  4).  Recovery  time  was  significantly  faster  for  the  ADI  ball  than  for  the  HUD  pitch  ladder. 
The  authors  recommend  placement  of  an  ADI  ball  in  a  prominent  position  in  the  F/A-18  pilot's  visual 
field.  They  also  discuss  other  options  which  would  provide  similar  information  in  the  pilot's  primary 
viewing  area. 

KEYWORDS:  attitude  displays,  aviation  (military),  display  design,  measurement, 

performance-based  measures,  pilot  aiding,  pilot  error,  subjective  measures 
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120  Klein,  G. A.  (1989a,  May).  Strategies  of  decision  making.  Military  Review,  69{5), 
56-64. 

Klein  uses  this  article  to  present  his  Recognitional  Decision  Making  (RDM)  theory  to  the  operational 
Army  officer  community  and  to  discuss  its  implications  to  their  enviromnent.  Basically,  he  suggests 
that  too  much  of  the  Army's  guidance  about  the  decision-making  process  assumes  that  analytical 
decision  making  involving  the  comparison  of  multiple  options  is  the  best  or  even  the  only  way  to  make 
decisions.  Klein  argues  that  in  the  decision  making  of  experts,  such  reliance  on  comparing  options  is 
not  common  and  may  not  be  desirable.  The  key  issue  in  many  of  these  settings  is  whether  the  situation 
assessment  formed  by  the  decision  maker  is  well  executed.  In  RDM  theory,  the  situation  assessment 
of  an  expert  will  directly  lead  to  a  workable  solution  path.  Klein  discusses  how  Army  staff  interactions 
and  training  can  be  de.signed  to  support  and  improve  this  approach  to  decision  making. 

KEYWORDS:  battlefield  SA,  cognitive  processes,  decision  aiding,  decision  making, 
expertise,  team  SA,  training 


121.  Klein,  G. A.  (1989b).  Recognition-primed  decisions.  In  W.B.  Rouse  (Ed.), 

Advances  in  man-machine  systems  research  (Volume  5,  pp.  47-92).  Greenwich, 
CN:  JAI  Press. 

This  paper  is  an  early  theoretical  treatment  of  the  Recognition-Primed  Decision  (RPD)  model. 
Reviewing  the  results  of  several  evaluations  of  real-world  decision  making,  Klein  suggests  that  the 
traditional  model  of  analytical  decision  making  by  the  comparison  of  alternatives  must  be  supplemented 
by  the  RPD  model,  The  RPD  model  suggests  that  the  decision  maker  forms  a  situation  assessment  of 
the  current  situation,  and  this  situation  assessment  is  directly  linked  with  a  potential  solution.  Unless 
mental  simulation  of  attempts  to  use  the  indicated  solution  prove  it  to  be  unworkable,  no  other  options 
are  typically  considered.  Increasing  expertise  or  time  pressure  will  increase  the  likelihood  that  a  RPD 
approach  will  be  utilized. 

KEYWORDS:  cognitive  processes,  decision  making,  expertise,  measurement,  mental 
models,  retrospective  measures,  subjective  measures 


122.  Klein,  G. A.  (1993,  April).  Naturalistic  decision  making:  Implications  for  design 
(CSERIAC  State  of  the  Art  Report  93-01).  Wright-Patterson  Air  Force  Base,  OH: 
Crew  System  Ergonomics  Information  Analysis  Center. 

This  report  reviews  the  basic  tenets  of  Klein's  Naturalistic  Decision  Making  (NDM)  model.  The 
emphasis  is  on  the  pragmatic  issues  of  using  NDM  models  to  help  guide  the  design  and  development  of 
human-machine  interfaces.  Topics  discussed  include  the  decision  strategies  commonly  used  by 
operators  that  emphasize  situation  assessment,  how  cognitive  task  analysis  can  be  used  to  define  the 
decision  requirements  of  operators,  and  the  potential  role  of  NDM  in  cognitive  systems  engineering  to 
design  interfaces  that  reflect  normal  cognitive  processes. 

KEYWORDS:  cognitive  processes,  decision  aiding,  decision  making,  expertise, 
measurement,  retrospective  measures,  subjective  measures 
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123.  Klein,  G.A.,  Calderwood,  R.,  Clinton-Cirocco,  A.  (1986).  Rapid  decision  making  on 
the  fire  ground,  in  Proceedings  of  the  Human  Factors  Society  30th  Annual 
Meeting  (Volume  1,  pp.  576-580).  Santa  Monica,  CA:  Human  Factors  Society. 

The  decision  making  procedures  used  by  skilled  fire  ground  commanders  (FCJCs)  was  studied.  Data  on 
the  process  were  collected  using  retrospective  reports  of  the  thought  processes  experienced  during 
actual  incidents,  These  data  were  collected  in  semi-structured  interviews.  The  data  were  then  analyzed 
by  looking  for  "decision  points"  (defined  as  "a  point  in  time  when  non-trivial  alternative  courses  of 
action  are  available"  (p.  576))  and  keeping  track  of  "situational  awareness"  (defined  as  "the  information 
and  cues  that  the  FGC  was  considering  at  the  time  of  each  event"  (p.  577)).  A  major  finding  of  the 
study  was  that  multiple  alternative  courses  of  action  were  rarely  weighed  against  each  other.  Instead  of 
comparing  alternatives,  the  FGC's  choices  were  most  commonly  based  on  their  situational  awareness  of 
the  current  situation  and  generating  an  analogous  situation  from  experience.  This  observation  of  the 
decision  making  process  led  the  author  to  forming  the  "recognition-primed  decision  (RPD)  model." 

KEYWORDS;  cognitive  processes,  decision  making,  definition,  expertise,  measurement, 
questionnaire/survey  data,  retrospective  measures,  subjective  measures, 
verbal  protocols 


124.  Klein,  G.A.,  Orasanu,  J.,  Calderwood,  R.,  and  Zsambok,  C.E.  (Eds.)  (1993), 
Decision  making  in  action:  Models  and  methods.  Norwood,  NJ:  Ablex. 

This  book  presents  23  chapters  that  review  the  development,  methods,  and  applications  of  naturalistic 
decision  making  theories.  Situation  assessment  is  a  key  component  of  many  of  the  models  and  theories 
presented. 

KEYWORDS:  cognitive  processes,  decision  aiding,  decision  making,  expertise 


125.  Klinger,  D.W.,  and  Gomes,  M.E.  (1993).  A  cognitive  systems  engineering 
application  for  interface  design.  In  Proceedings  of  the  Human  Factors  and 
Ergonomics  Society  37th  Annual  Meeting  (Volume  1,  pp.  16-20).  Santa  Monica, 
CA:  Human  Factors  and  Ergonomics  Society. 

This  article  describes  using  a  Cognitive  Task  Analysis  procedure  to  identify  problems  faced  by  AWACS 
Weapons  Directors  (WDs)  and  then  to  guide  development  of  solutions  to  those  problems.  Five 
problem  areas  were  identified  for  the  current  system:  attention,  memory,  situational  awareness, 
workload,  and  decision  making.  An  alternative  display  concept  was  developed  and  tested  in  a 
high-fidelity  simulation  of  the  WD's  task.  Based  on  improvements  in  various  measures  of  effectiveness 
of  the  WD's  perfomtance,  the  authors  contend  that  SA  was  improved. 

KEYWORDS:  attention,  aviation  (military),  cognitive  processes,  combat  simulation, 
decision  making,  display  design,  measurement,  performance- based 
measures,  pilot  aiding 
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126,  Kuchar,  J.K.,  and  Hansman,  R.J.  (1993),  An  exploratory  study  of  plan-view 
terrain  displays  for  air  carrier  operations.  The  International  Journal  of  Aviation 
Psychology,  3,  39-54. 

This  research  investigated  the  utility  and  possible  designs  for  Terrain  Situation  Awareness  Displays 
(TSA).  TSA  was  described  by  the  authors  (p.  40)  as  involving,  "the  presentation  of  terrain  information 
in  a  manner  that  allows  the  pilot  to  create  a  mental  model  of  the  topography  surrounding  the  aircraft's 
planned  trajectory."  Two  formats  for  presenting  terrain  information  on  the  TSA  were  compared;  (1)  a 
spot-elevation  display  indicating  local  peak  elevations,  and  (2)  a  smoothed-contour  display  of  local 
terrain.  The  smoothed-contour  display  supported  better  hazard  recognition  performance.  The  pilots 
were  also  asked  to  rate  the  two  displays  on  a  five-point  SA  scale  at  the  completion  of  the  evaluation. 
There  was  a  statistically  significant  difference  in  the  ratings  for  the  two  displays.  The  smoothed 
contour  display  was  rated  as  providing  superior  TSA. 

KEYWORDS:  aviation  (civilian),  cognitive  processes,  display  design,  measurement, 
mental  models,  performance-based  measures,  pilot  aiding,  subjective 
measures 


127.  Kuperman,  G.G.,  and  Wilson,  D.L.  (1988).  The  design  of  a  tactical  situation 
display.  In  Proceedings  of  the  Human  Factors  Society  32nd  Annual  Meeting 
(Volume  1,  pp.  111-115).  Santa  Monica,  CA:  Human  Factors  Society, 

The  paper  reviews  the  development  of  a  Tactical  Situation  Display  (TSD)  for  a  conceptual 
next-generation  strategic  bomber.  Such  a  display  was  considered  necessary  in  order  to  allow  the  crew 
to  deal  effectively  with  changing  missions  in  real-time  while  having  only  incomplete  information. 
Preliminary  evaluation  of  some  of  the  display  concepts  was  discussed  in  Marshak,  Kuperman,  Ramsey, 
and  Wilson  (1987). 

KEYWORDS:  aviation  (miiitary),  display  design,  pilot  aiding 


128,  Lesser,  R,  (1993,  June).  Avionics  2000  --  "May  I  help  you?"  Defense 
Electronics,  25(7),  37-40. 

The  article  discusses  two  research  programs  designed  to  evaluate  the  role  artificial  intelligence  can  play 
in  improving  pilot  SA.  The  two  programs  are  the  Air  Force  Pilot's  Associate  (PA)  program  and  the 
Army's  Rotorcraft  Pilot's  Associate  (RPA)  program.  In  both  cases,  AI  is  being  tested  as  a  means  to 
filter  and  integrate  mission  relevant  information  and  present  it  to  the  pilot  in  the  most  effective  manner. 
One  goal  of  the  program  is  to  improve  pilot  SA.  A  preliminary  test  of  the  PA  system  with  F-15  pilots 
generated  "enthusiastic"  support  for  the  system.  Some  elements  of  PA  are  being  implemented  in  the 
F-22  .vionics  package. 

KE'  VORDS:  aviation  (military),  avionics,  data  fusion,  helicopter,  measurement,  pilot 
aiding,  subjective  measures 
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129.  Lippert,  F.G.,  Shechter,  J.,  and  Burke,  J.  (1989),  Pilot/surgeon  inflight  decision 
making:  A  study  of  the  integration  of  aviation  and  operating  room  cognitive  skills. 
In  Proceedings  of  the  Fifth  internationai  Symposium  on  Aviation  Psychology 
(Volume  1,  pp.  366-370).  Columbus:  The  Ohio  State  University. 

hi:;  paper  discussed  SA  as  one  element  that  flying  an  aircraft  and  doing  surgery  might  have  in 
junimon.  The  similarities  and  differences  between  the  two  settings  in  terms  of  SA  are  discussed.  The 
SA  abilities  of  pilots  that  were  also  practicing  physicians  were  assessed  in  a  dynamic  scenario  of 
aviation  events.  The  pilot/physicians  were  asked  to  give  verbal  summaries  of  their  situation 
assessments  at  periodic  points  in  the  scenario.  These  reports  were  then  compared  to  the  true  situation 
to  provide  one  measure  of  S  A.  Another  measure  of  SA  was  provided  by  observer  pilot  ratings.  The 
authors  report  that  the  pilot^physicians  were  much  less  likely  to  investigate  and  solve  SA  discrepancies 
in  the  aircraft  than  they  would  in  the  operating  room.  Based  on  this,  the  authors  conclude  that  there  is 
little  automatic  transfer  of  the  decision  making  skills  of  physicians  to  aviation. 

KEYWORDS:  aviation  (civilian),  cognitive  processes,  decision  making,  measurement, 
subjective  measures,  verbal  protocols 


130.  Lum,  Z.A.  (1993,  September),  The  path  to  heightened  awareness.  Journal  of 
Electronic  Defense,  76(9),  43-44,  46,  48. 

The  author  identifies  SA  as  a  term  that  "rings  like  a  mantra  through  the  halls  of  the  Pentagon"  (p.  43). 
To  help  pin  dovm  the  concept,  the  author  examines  three  examples  of  systems  designed  to  improve 
combat  SA;  one  system  from  each  of  the  three  military  services.  All  three  systems  highlight  the 
potential  for  new  sensor,  communication,  and  data  processing  technology  to  provide  combatants  with  a 
much  larger  and  more  complete  picture  of  the  combat  environment.  The  possibility  that  threat 
assessment  and  tactical  option  selection  could  be  automated  is  also  discussed.  Finally  it  is  noted  that 
(p.  48),  "In  the  end,...  the  trick  will  be  not  to  supply  enough  information  to  the  user,  but  to  manage  the 
gouts  of  data  in  a  usable  form." 

KEYWORDS:  aviation  (military),  avionics,  battlefield  SA,  decision  aiding,  naval  SA,  pilot 
aiding,  target  identification 
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131,  Marshak,  W.P.,  Kupernian,  G.,  Ramsey,  E.G.,  and  Wilson,  D.  (1987).  Situational 
awareness  in  map  displays.  In  Proceedings  of  the  Human  Factors  Society  31st 
Annual  Meeting  (Volume  1,  pp.  533-535).  Santa  Monica,  CA:  Human  Factors 
Society. 

Situation  awareness  was  assessed  for  two  different  map  display  fonnats:  a  track-up  moving  map 
display,  and  a  north-up  moving  plane  display.  A  situation  awareness  metric  was  developed.  It 
consisted  of  summing  the  absolute  percent  error  across  a  series  of  probe  questions  asked  at  random 
times  while  the  subjects  were  monitoring  a  mission  on  one  of  the  displays.  Despite  extremely  noisy 
dat'  the  memory  probe  situation  awareness  metric  was  successful  in  discriminating  between  flight 
pa  )f  different  levels  of  complexity  and  the  two  types  of  displays.  Additionally,  subjects  were  asked 
to  mt  a  preference  questionnaire  following  the  experiment.  The  preference  data  dissociated  from 
the  ciformance  data.  Whereas  performance  was  better  with  the  moving  map  display,  the  subjects 
preferred  the  moving  plane  display.  The  authors  conclude  that  use  of  subjective  evaluation  is  risky 
unless  the  subjective  results  are  supported  by  appropriate  performance  data. 

KEYWORDS;  aviation  (military),  display  design,  measurement,  memory  probe  measures, 
pilot  aiding,  questionnaire/survey  data,  retrospective  measures,  subjective 
measures 


132.  Masters,  R.M.,  McTaggart,  T.E.,  and  Green,  G.L.  (1986).  Air-to-air  F-1 S/F-1 6 
identification  analysis  methodology  -  Management  summary  (Tech.  Report  No. 
ASD-TR-87-5004).  Wright-Patterson  Air  Force  Base,  OH:  Combat  Identification 
System  Program  Office.  (AD-B1 08209) 

This  report  is  an  unclassified  management  summary  of  the  objectives,  methodology,  analytical  process 
and  application  of  an  NCTR  (Non-Cooperative  Target  Recognition)  Identification  study.  Situation 
Awareness  is  defined  here  as  "the  ability  to  envision  the  current  and  near-term  disposition  of  both 
friendly  and  enemy  forces."  The  study,  conducted  for  the  Air  Force  Combat  Identification  System 
Program  Office,  analyzed  ID  requirements  for  F-1 5s  and  F-1 6s  in  the  1990s.  The  specific  focus  was 
composite  fire  control/ID  system  performance  parameters  for  a  future  integrated  sensor  suite. 

KEYWORDS:  aii'-to-air  mission,  aviation  (military),  avionics,  data  fusion,  definition,  target 
identification 


133.  McKinley,  R.L.,  Erikson,  M.A.,  and  D'Angelo,  W.R.  (1994).  3-Dimensional 

auditory  displays;  Development,  applications,  and  performance.  Aviation,  Space, 
and  Environmental  Medicine,  65(5,  Supplement),  A31-A38. 

This  article  reviews  both  laboratory  and  in-flight  evaluation  of  a  3-D  audio  display  system.  The 
in-flight  tests  were  conducted  in  a  AV-8B  "Harrier"  aircraft.  Summarizing  the  results  of  the  in-flight 
test,  the  authors  state  (p.  A31):  "Flight  test  results  demonstrated  successful  cued  target  acquisition,  a 
subjective  decrease  in  target  acquisition  times,  a  subjective  improvement  in  speech  intelligibility,  a 
subjective  increase  in  situational  awareness,  and  a  subjective  decrease  in  pilot  workload." 

KEYWORDS.  3'D  audio  displays,  aviation  (military),  display  design,  flight  test, 
measurement,  subjective  measures,  workload 
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134.  Melanson,  D.,  Curry,  R.E.,  Howell,  J.D.,  and  Connelly,  M.E.  (1973).  The  effects 
of  communications  and  traffic  situation  displays  on  pilots  awareness  of  traffic  in 
the  terminal  area.  In  Proceedings  of  the  Ninth  Annual  Conference  on  Manual 
Control  (pp.  25-39).  Cambridge:  Massachusetts  Institute  of  Technology. 

The  authors  were  asked  to  determine  the  "assurance"  of  pilots  using  a  proposed  Traffic  Situation 
Display  (TSD)  with  or  without  "shared"  communications  that  allowed  monitoring  of  communications 
of  air  traffic  control  with  other  aircraft.  The  authors  equated  "assurance"  with  awareness  and  defined 
awareness  as  having  four  components  in  their  evaluation:  (1)  knowledge  of  own  position,  (2) 
knowledge  of  the  positions  of  others,  (3)  ability  to  predict  the  future  evolution  of  the  traffic  situation, 
and  (4)  ability  to  choose  an  appropriate  escape  route.  Awareness  was  measured  by  a  "stop-action  quiz" 
that  involved  freezing  and  blanking  the  displays  unexpectedly  during  a  trial  and  asking  the  subjects  to 
fill  in  a  map  with  the  relevant  information.  Mental  workload  was  assessed  by  an  auxiliary  self-paced 
task  to  be  performed  along  with  the  simulated  flight  control.  Both  the  stop-action  quiz  and  the 
workload  measure  were  sensitive  to  some  of  the  experimental  manipulations. 

KEYWORDS;  3'''  traffic  control,  aviation  (civilian),  definition,  display  design, 
measurement,  memory  probe  measures,  pilot  aiding,  workload 


135.  Metalis,  S.A.  (1993).  Assessment  of  pilot  situational  awareness:  Measurement  via 
simulation.  In  Proceedings  of  the  Human  Factors  and  Ergonomics  Society  37th 
Annual  Meeting  (Volume  1,  pp.  113-117).  Santa  Monica,  CA:  Human  Factors  and 
Ergonomics  Society. 

The  primary  objective  of  this  paper  is  to  present  an  approach  for  measuring  SA  in  cockpit  design 
evaluations.  Concerned  about  the  lack  of  a  good  definition  of  SA  ("they  tend  to  use  one  mentalistic 
term  to  define  another"  (p,  1 14)),  the  author  suggests  that  efforts  focus  on  agreeing  about  the 
operationalisms  associated  with  influencing  SA.  According  to  the  author  (p.  1 13),  "SA  appears  to 
involve  the  development  and  maintenance  of  a  highly  dynamic  mental  representation  of  critical  aspects 
of  the  flying  environment."  This  representation  is  assumed  to  allow  the  pilot  to  make  and  implement 
timely  decisions  with  little  conscious  effort.  As  an  approach  for  measurement,  the  author  suggests  an 
analogy  to  benchmarking  a  computer.  Applied  to  the  SA  evaluation  of  competing  cockpit  designs,  this 
would  entail  performing  "benchmark"  missions  with  each  cockpit  in  a  medium  fidelity  simulator  under 
increasing  levels  of  difficulty.  Difficulty  of  most  missions  could  be  increased  by  requiring  higher 
airspeeds.  During  the  course  of  these  simulated  missions  a  variety  of  SA  relevant  data  would  be 
collected.  Objective  data,  such  as  flight  performance  and  reaction  to  unexpected  events,  would  be 
combined  with  subjective  data,  such  as  ratings  or  knowledge  elicitation.  These  data  are  then  weighted 
and  combined  by  multivariate  statistics  to  produce  an  aggregate  S  A  score.  At  the  time  of  the  writing  of 
this  paper  the  author  had  not  collected  sufficient  data  to  assess  the  validity  and  reliability  of  the 
approach,  but  reported  that  the  preliminary  research  seemed  promising. 

KEYWORDS;  aviation  (military),  cognitive  processes,  combat  simulation,  decision  making, 
definition,  measurement,  memory  probe  measures,  mental  models, 
performance-based  measures,  SAGAT,  subjective  measures 
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136.  Mogford,  R.H.  (1994).  Mental  models  and  situation  awareness  in  air  traffic  control. 
In  R.D.  Gilson,  D.J.  Garland,  and  J.M,  Koonce  (Eds.),  Situational  awareness  in 
complex  systems:  Proceedings  of  a  CAHFA  Conference  (pp.  199-207).  Daytona 
Beach,  FL:  Embry-Riddle  Aeronautical  University  Press. 

The  author  offers  a  definition  of  SA  (p.  199):  "A  proposed  definition  of  situation  awareness  is,  for  any 
given  system,  the  transient  contents  of  awareness  as  structured  and  supported  by  an  underlying  mental 
model."  In  this  view  the  "picture"  provided  by  SA  consolidates  the  sensory  impression  of  the  situation 
for  processing  by  a  mental  model.  A  research  project  was  conducted  to  determine  if  the  quality  of  an 
air  traffic  control  student's  SA,  as  measured  by  a  memory  probe  procedure,  is  related  to  the  student's 
overall  performance  as  an  air  traffic  controller,  as  rated  by  expert  instructors.  The  memory  probe 
procedure  extracted  precise  information  about  the  location  and  status  of  aircraft  in  a  simulated  air 
traffic  control  scenario  following  a  freeze  of  the  simulation.  The  students'  performance  on  a  variety  of 
parameters  were  then  tested  as  predictors  of  the  students'  ratings  by  expert  air  traffic  control 
instructors  who  observed  the  students  in  normal  air  traffic  control  simulations.  Several  of  the  memory 
probe  variables  were  found  to  be  statistically  related  to  the  instructor  ratings.  This  was  interpreted  as 
supporting  the  contention  that  good  SA  of  basic  aircraft  parameters  in  the  airspace  forms  a  basis  for 
good  air  traffic  control  in  trainees. 

KEYWORDS:  3'''  traffic  control,  cognitive  processes,  definition,  measurement,  memory 
probe  measures,  mental  models,  training 


137.  Morishige,  R.I.,  and  Retelle,  J.  (1985,  October).  Air  combat  and  artificial 
intelligence.  Air  Force  Magazine,  55(10),  91-93. 

This  paper  reviews  the  Air  Force's  Pilot  Associate  (PA)  program.  As  described  by  the  authors  (p.  92), 
PA  "seeks  to  reverse  the  trend  towards  proliferation  of  systems  in  fighter  aircraft  that  generate  massive 
amounts  of  data  and  dump  it  on  the  pilot."  The  PA  will  accomplish  this  by  using  artificial  intelligence 
to  integrate,  prioritize,  and  filter  the  data  and  then  present  the  most  significant  information  to  the  pilot. 
The  .lain  justification  for  the  PA  is  that  it  will  improve  the  pilot's  SA;  that  is,  the  pilot's  "awareness  of 
conuitions  and  threats  in  the  immediate  surroundings"  (p.  92). 

KEYWORDS:  aviation  (military),  avionics,  definition,  pilot  aiding 


138.  Morrocco,  J.D.  (1991,  March  25).  McDonnell  F/A-18E/F  offers  increased  range, 
endurance.  Aviation  Week  &  Space  Technology.  734(12),  25-26. 

The  article  discusses  possible  design  concepts  for  the  F/A-18E/F  aircraft.  Among  the  possible  changes 
to  tf  F-18  in  the  E/F  model  is  the  replacement  of  the  5X5  inch  multipurpose  color  display  located  in 
the  liter  of  the  cockpit  display  area  with  an  8  X  8  inch  flat  panel  color  tactical  situation  display.  This 
disp  :  would  be  capable  of  displaying  a  moving  map  as  well  as  tactical  data.  According  to  the  article 
(p. ;  ),  "Electronics  Lead  Engineer  Bruce  K.  Easterling  said  the  system  would  provide  pilots  with 
gre<  r  situational  awareness  by  fusing  more  information  together  in  a  single,  large  display." 

KEi  WORDS:  aviation  (military),  data  fusion,  display  design,  pilot  aiding 
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139.  Mosier,  K.L.,  and  Chidester,  T.R.  (1991).  Situation  assessment  and  situation 

awareness  in  a  team  setting.  In  Designing  for  Everyone:  Proceedings  of  the  11th 
Congress  of  the  internationai  Ergonomics  Association  (Voiume  1,  pp.  798-800). 
Paris,  France:  Tayior  &  Francis. 

This  paper,  along  with  4  other  SA  related  papers  presented  at  the  Congress,  were  reprinted  in  Taylor 
(1991)  and  is  reviewed  under  that  listing. 


KEYWORDS:  aviation  (civilian),  cockpit  resource  management  (CRM),  decision  making, 

measurement,  performance-based  measures,  team  decision  making,  team  SA 


140.  Nagei,  D.C.  (1988).  Human  error  in  aviation  operations.  In  E.L.  Wiener  and  D.C. 
Nagel  (Eds.),  Human  factors  in  aviation  (pp.  263-303).  San  Diego,  CA:  Academic 
Press. 

The  author  discusses  the  causes  and  possible  remedies  for  accidents  in  commercial  aviation.  As  one  of 
the  key  causes  for  accidents,  the  author  points  to  inefficient  information  transfer  to  the  pilot  from 
displays,  crew,  and  radio  communications  resulting  in  breakdowns  of  pilot  situational  awareness.  The 
author  cites  an  earlier  NASA  study  of  the  Aviation  Safety  Reporting  System  (ASRS)  database  that 
stated  that  more  than  73%  of  more  than  12,000  incidents  in  commercial  aviation  were  associated  with 
an  information  transfer  problem.  The  author  also  suggests  that  increasing  use  of  automation  on  the 
flight  deck  is  producing  some  problems  (p.  300),  "As  pilots  are  removed  from  an  active  role  in  flying 
the  aircraft,  more  and  more  incidents  that  can  only  be  termed  'loss  of  situational  awareness’  are 
reported." 

KEYWORDS;  aviation  (civilian),  avionics,  measurement,  pilot  error,  retrospective 
measures,  subjective  measures 


113 


141.  Naish,  P.L.N.  (1990,  April).  The  simulation  of  localized  sounds  for  improved 
situational  awareness.  In  AGARD-CP-478,  Situational  Awareness  in  Aerospace 
Operations  (pp.  12-1  to  12-S).  Neuilly  Sur  Seine,  France:  Advisory  Group  for 
Aerospace  Research  &  Development.  (AD-A223939) 

Naish  reports  on  two  separate  studies  conducted  to  look  at  the  potential  gain  from  adding  a  localization 
dimension  to  improve  processing  of  audio  cues  in  the  cockpit.  The  first  study  was  somewhat  of  a 
sophisticated  Stroop  task,  where  subjects  were  given  four  representations  of  “left,  right  or  centre." 

One  of  these  words  was  written  on  a  computer  screen,  and  placed  either  at  the  left,  right,  or  centre  of 
the  screen.  Simultaneously  a  word  (left,  right,  or  centre)  was  spoken  over  headphones,  and  that  word 
w(  d  seem  to  come  from  either  the  left,  right,  or  centre  of  the  headphones.  Subjects  were  told  which 
cu  0  attend  to,  and  performance  (RT)  with  0,  1,  2,  or  3  incongruent  cues  was  analyzed.  Naish  found 
thi..  t  /en  one  incongruent  cue  significantly  increased  reaction  time  for  this  task;  in  addition, 
incongruent  cues  of  the  same  modality  caused  a  significantly  higher  increase  in  reaction  time  than  cues 
of  different  modality.  In  the  second  study,  subjects  performing  a  tracking  task  carried  out  a  secondary 
task  to  identify  which  of  eight  LEDs  was  lit  on  a  hoop  around  their  heads.  There  were  three  conditions 
under  which  the  lights  were  presented;  with  no  auditory  warning,  with  a  centralized  bleep,  and  with  a 
localized  warning  as  to  where  the  LED  would  be  lit.  Response  times  to  type  in  the  LED  number  were 
significantly  faster  following  localized  warnings  than  following  central  bleeps.  There  were  considerable 
inter-subject  differences.  Naish  concludes  from  these  studies  that  synthetic  direction  cues,  even  crude 
ones,  can  aid  in  acquiring  spatial  information. 

KEYWORDS;  audio  displays,  display  design,  measurement,  performance-based 
measures,  pilot  aiding 


142.  Nordwall,  B.D,  (1989,  May  8).  Pilots  sense  attitude  with  peripheral  vii.  lon  using 
new  Garrett  display.  Aviation  Week  &  Space  Technology,  730(19),  97,  99. 

This  article  discusses  Garrett  of  Canada's  Peripheral  Vision  Display  (PVD).  The  PVD  is  a  display 
system  that  uses  a  laser  to  place  a  horizon  line  across  the  entire  cockpit  display  area.  It  is  intended  to 
provide  attitude  information  through  peripheral  vision  and  to  reduce  the  probability  that  spatial 
disorientation  will  cause  aircraft  accidents.  A  report  on  helicopter  tests  conducted  by  the  Naval  Air 
Test  Center  stated  that  the  PVD  improved  SA  while  reducing  workload.  However,  the  results  of  a  test 
program  to  evaluate  the  PVD  in  F-16s  were  unsatisfactory.  The  poor  results  may  have  been  due  to 
difficulties  in  finding  an  appropriate  mounting  location  and  other  physical  attributes  of  the  F-16 
cockpit.  Also,  pilots  may  require  specific  training  on  the  use  of  peripheral  vision  for  using  the  display. 
Some  pilots  had  a  tendency  to  stare  directly  at  the  display. 

KEYWORDS;  attitude  displays,  aviation  (military),  display  design,  flight  test,  helicopter, 
measurement,  pilot  aiding,  subjective  measures 
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143.  Nordwall,  B.D.  (1990,  April  23).  Competing  ATF  proposals  to  have  different 
avionics  suite  designs.  Aviation  Week  &  Space  Technoiogy,  /32(16),  55-58. 

The  development  of  the  avionics  suite  for  the  Air  Force's  Advanced  Tactical  Fighter  (ATF)  is  described. 
One  primary  goal  of  the  avionics  package  is  to  integrate  the  use  of  active  and  passive  sensors  to 
provide  the  pilot  with  situational  awareness  information  while  keeping  detectable  emissions  to  a 
minimum. 

KEYWORDS:  aviation  (military),  avionics,  data  fusion,  pilot  aiding 


144.  Nordwall,  B.D.  (1991,  June  10).  Slow  pace  on  flat  displays  hinders  cockpit 
advances.  Aviation  Week  &  Space  Technoiogy,  134(23).  61,  64. 

The  article  discusses  modem  display  technologies  that  can  be  applied  to  the  military  aircraft  cockpit. 
Filling  the  needs  of  the  military  pilot  is  described  as  difficult  (p.  61):  "Complicating  the  problem  for 
designers  is  the  fact  that  a  pilot  needs  situational  awareness  of  both  'the  big  picture'  and  his  personal 
tactical  visual  situation."  Combining  large-area  "big  picture"  displays  and  helmet-mounted  displays  is 
presented  as  a  possible  solution.  Suitable  helmet-mounted  displays  appear  to  be  within  the  grasp  of 
current  technology,  but  the  big  picture  display  needs  funher  development.  However,  Air  Force 
simulator  tests  reportedly  demonstrated  an  almost  50%  improvement  in  fighter  pilot  kill  ratios  with  the 
advanced  cockpit  technologies. 

KEYWORDS:  aviation  (military),  big  picture  displays,  combat  simulation,  display  design, 
helmet-mounted  display  (HMDs),  measurement,  performance-based 
measures,  pilot  aiding 


145.  Nordwall,  B.D.  (1991,  August  5).  New  data  link  gave  F-14  pilots  major  advantage 
in  Desert  Storm.  Aviation  Week  &  Space  Technoiogy,  135(5),  46-47. 

The  article  reviews  a  new  data  linking  capability  for  the  F-14  aircraft  that  was  operational  during  the 
Gulf  War.  The  digital  data  link  allowed  F-14s  to  directly  share  tactical  information  with  each  other 
without  the  need  of  a  intermediary  command  and  control  platform.  The  tactical  display  showed  the 
location  of  other  F-14s  and  their  radar  targets.  According  to  Cdr.  Thomas  Enright,  commander  of  a 
F-14  squadron  during  the  war  (quoted  on  p.  46),  "The  result  was  fantastic  situational  awareness." 
Enright  further  noted  that  the  system  was  especially  useful  for  "deconflicting."  The  term 
"deconflicting"  refers  to  the  process  of  sorting  out  the  identities  of  the  various  aircraft  to  avoid  firing 
on  a  friendly  aircraft. 

KEYWORDS:  aviation  (military),  avionics,  data  link,  display  design,  flight  test,  fratricide, 
pilot  aiding,  target  identification 
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146,  Nordwall,  B.D.  (1992,  October  19).  ITT  system  will  give  pilots  better  threat 
information-  Aviation  Week  &  Space  Technology,  ?37(16),  41. 

ITT  is  developing  a  system  to  increase  pilot  SA  in  Amiy  helicopters,  The  concept  is  called  the  Radar 
Deception  and  Jamming  (RD&J)  system,  and  is  intended  to  increase  SA  by  providing  the  pilot  with 
exact  information  about  the  location  of  possible  threats.  The  RD&J  system  is  designed  to  improve  SA 
by  using  a  three-step  process:  process  data  fusion  (combining  data  from  on-board  sensors  with 
previously  programmed  knowledge),  sensor  fusion  (using  intelligence  data  from  a  data  link  system), 
and  information  display  (the  prioritized  threat  data  will  be  displayed  on  a  digital  terrain  map). 

KEYWORDS:  avionics,  data  fusion,  data  link,  display  design,  helicopter,  pilot  aiding 


147.  Nordwall,  B.D.  (1993,  July  5).  EW  goal  is  improved  situational  awareness. 
Aviation  Week  &  Space  Technology,  733(1),  59. 

The  article  suggests  that  electronic  warfare  (EW)  should  play  a  role  in  future  improvements  of  Air 
Force  pilot  SA.  These  improvements  are  expected  to  develop  from  fusing  data  from  off-board  sensors 
with  the  aircraft's  own  sensor  information.  Programs  to  test  potential  sources  of  information  are 
discussed.  The  initial  goal  is  to  "transmit  all  the  information  to  a  tactical  aircraft"  (p.59).  However,  the 
author  also  points  out  that  (p.59),  "The  question  of  how  to  present  the  information  without 
overloading  the  pilot  has  not  been  determined."  Two  approaches  to  solving  this  information 
presentation  problem  are  mentioned:  "feed  all  information  to  every  aircraft  and  let  the  pilot  select  what 
he  needs"  or  "have  the  software  for  a  specific  mission  automatically  select  the  EW  information  the  pilot 
would  need." 

KEYWORDS:  aviation  (military),  avionics,  data  fusion,  data  link,  pilot  aiding 


148.  Norman,  D. A.  (1993).  Things  that  make  us  smart.  Reading,  MA: 

Addison-Wesley. 

In  a  discussion  of  "Distributed  Cognition,"  Norman  considers  the  possible  benefits  of  traditional 
workplaces  relative  to  proposed  improvements.  One  example  discussed  is  the  traditional  commercial 
airliner  cockpit  with  large  linked  controls  versus  more  modern  variants.  Norman  contends  that  the 
traditional  cockpit  design  created  unappreciated  and  perhaps  unintended  channels  of  communication 
that  were  nonetheless  valuable.  For  example,  the  large,  linked  control  yokes  made  it  instantly  obvious 
who  was  controlling  the  aircraft  and  what  that  person  was  doing.  Modern  side  stick  controllers  may 
destroy  this  channel  of  communication.  Norman  associates  this  with  SA  (p.  142),  "The  critical  thing 
about  doing  shared  tasks  is  to  keep  everyone  informed  about  the  complete  state  of  things.  The 
technical  term  for  this  is  situation  awareness;  Each  pilot  or  member  of  the  control  team  must  be  fully 
aware  of  the  situation,  of  what  has  happened,  what  is  planned."  In  general,  Norman  cautions  against 
making  improvements  to  team  environments  that  seem  to  improve  efficiency,  but  may  inadvertently 
degrade  communication  among  team  members. 

KEYWORDS:  aviation  (civilian),  cognitive  processes,  decision  making,  team  decision 
making,  pilot  aiding,  team  SA 
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149.  North,  D.M.  (1990,  February  26).  Aviation  Week  editor  flies  Soviet-ba.sed  MiG-29 
fighter.  Aviation  Week  &  Space  Technology,  132(9),  36-38,  42-45,  48. 

The  article  reviews  an  evaluation  flight  of  a  MiG-29  fighter  by  an  Aviation  Week  editor.  For  the  most 
part,  the  author  was  impressed  by  the  aircraft.  However,  he  had  the  following  comments  about  the 
cockpit  design  (p.  38),  "The  first  impression  of  the  MiG-29  cockpit  is  that  it  is  comparable  to  the 
late- 1950s  McDonnell  Douglas  F-4  aircraft  with  analog  instruments  with  round  dials.  The  MiG-29 
instrument  panel  does  not  contain  the  digital  and  cathode-ray-tube-based  displays  found  in  m.'iny 
current  Western  fighters,  such  as  the  17A-18  and  F-IS.  The  instruments  and  displays  suggest  that  the 
Soviets  do  not  pay  as  much  attention  to  situational  awareness  for  the  pilot  as  do  Western  designers." 

KEYWORDS:  aviation  (military),  display  design,  flight  test 


150.  North,  D.M.  (1990,  September  24).  Aviation  Week  editor  flies  top  Soviet 
interceptor.  Aviation  Week  &  Space  Technology,  133(^3),  32-35,  38-39,  41. 

The  article  reviews  an  evaluation  flight  of  the  Soviet  Su-2V  fighter  aircraft  by  m  Aviation  Week  editor. 
The  editor  was  the  first  American  pilot  to  fly  the  aircraft.  In  most  respects  the  Su-27  was  judged  to  be 
competitive  with  current  U.S.  fighters.  However,  the  article  notes  (p.  35),  "But  the  lack  of  computer 
and  system  integration  technology  places  the  Soviet  pilot  at  a  disadvantage  in  terms  of  situation 
awareness." 

KEYWORDS:  aviation  (military),  avionics,  flight  test 


151.  North,  D.M.  (1991,  March  25).  Lavi  TD  cockpit  reflects  pilots'  combat  experience. 
Aviation  Week  &  Space  Technology,  134(^2),  46-47,  51-53. 

Based  on  an  evaluation  flight,  this  article  discusses  the  cockpit  design  of  the  Israeli  Lavi  Technology 
Demonstrator  (TD)  largely  in  terms  of  effects  on  pilot  SA.  The  importance  and  source  of  SA  is  stated 
in  no  uncertain  terms  (p.  46),  "Situational  awareness  is  one  of  the  most  important  elements  in  aerial 
combat,  be  it  air-to-air  or  air-to-ground  operations.  Ultimately,  good  human  engineering  is  what 
produces  this  key  feature."  More  specifically,  the  aircraft's  cockpit  integration  and  display  designs  are 
cited  as  reducing  pilot  workload  and  enhancing  SA.  The  editor  ranks  the  ID's  success  in  reducing 
workload  and  enhancing  SA  as  far  superior  to  the  Soviet  Su-27  and  MiG-29  designs. 

KEYWORDS:  aviation  (military),  avionics,  display  design,  flight  test,  pilot  aiding, 
workload 
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152,  North,  D.M.  (1991,  July  15).  AMX  fills  air-to-ground  role  with  room  for  mission 
growth.  Aviation  Week  &  Space  Technoiogy,  135{2),  36-37,  40-41,  43-45. 

The  article  reviews  an  evaluation  flight  of  the  AMX  aircraft  conducted  by  diU  Aviation  Week  editor.  The 
AMX  is  an  inexpensive  ground  attack  aircraft  developed  by  an  Italian/Brazilian  team.  SA  was  used  to 
describe  the  value  of  good  visibility  during  landing  procedures  (p.  44),  "The  all-around  visibility  and  the 
18-deg.  over-the-nose  visibility  allowed  for  excellent  situational  awareness  in  the  traffic  pattern  of  the 
joint  military/commercial-use  airport," 

KEYWORDS:  aviation  (military),  flight  test,  pilot  aiding 


153.  North,  D.M.  (1992,  March  23).  Airbus  pilot  training  center  stresses  task-sharing, 
good  communications  as  key  to  flying  advanced  aircraft.  Aviation  Week  &  Space 
Technology,  136(^2),  63-64. 

The  article  discusses  the  aircrew  integrated  management  (AIM)  course  developed  by  the  Airbus 
Training  Center.  The  course  is  primarily  designed  to  educate  commercial  aircrews  about  cockpit 
resource  management  (CRM).  The  AIM  manual  defines  SA  as  (p.  64),  "an  accurate  perception  of  the 
factors  and  conditions  that  aflfect  the  aircraft  and  flight  crew  during  a  specific  period  of  time."  The 
course  describes  barriers,  problems  in  crew  coordination,  and  ineffective  communications  as 
contributors  to  poor  crew  SA.  Descriptions  and  illustrative  exercises  of  possible  ways  to  build  SA  are 
also  presented  in  the  course.  Eleven  clues  to  loss  of  SA  are  listed. 

KEYWORDS:  aviation  (civilian),  cockpit  resource  management  (CRM),  definition,  training 


154.  North,  D.M.  (1992,  July  13).  A340  handling,  cockpit  design  improve  on 

predecessor  A320.  Aviation  Week  &  Space  Technoiogy,  137(2),  38-41,  44-45. 

During  an  evaluation  flight  of  the  Airbus  A340,  m  Aviation  Week  editor  described  the  excellent  outside 
visibility  and  control  panel  layout  as  contributing  to  good  pilot  SA. 

KEYWORDS:  aviation  (civilian),  display  design,  flight  test,  pilot  aiding 


155.  O'Hare,  D.  (1992).  The  "artful"  decision  maker:  A  framework  model  for 

aeronautical  decision  making.  The  International  Journai  of  Aviation  Psychology,  2, 
175-191. 

The  article  presents  a  model  of  aeronautical  decision  making  (/\DM).  The  model  emphasizes  the  role  of 
situation  awareness  in  decision  making  and  planning.  As  O'Hare  puts  it  (p,  182),  "Most  routine 
decision  making  arises  directly  from  situation  awareness,  mapping  directly  to  response  selection." 
Situation  awareness  also  plays  a  significant  role  in  the  review  of  proposed  approaches  for  improving 
ADM. 

KEYWORDS:  cognitive  processes,  decision  aiding,  decision  making 
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156.  O'Lone,  R.G.  (1992,  March  23).  Boeing  works  to  keep  pilot  in  decision  loop  on 
board  advanced  777.  Aviation  Week  &  Space  Technology.  136(M),  61. 

The  article  describes  the  automation  strategy  being  employed  by  Boeing  in  the  design  of  the  777 
aircraft.  Brian  Kelly,  a  Boeing  777  flight  deck  specialist  is  quoted  as  describing  the  approach  as  follows 
(p.  Cl),  "We  try  to  automate  in  such  a  way  that  the  pilot  remains  in  the  decision  loop  and  the  pilot's 
awareness  of  the  situation  is  preserved  and  even  enhanced."  This  goal  is  accomplished  by  using  an 
automation  strategy  that  requires  the  automation  to  inform  the  crew  of  planned  action  and  getting  the 
crew  consent  before  executing  it.  The  hope  is  to  take  some  workload  out  of  the  tasks  but  to  keep  the 
pilot  in  the  loop. 

KEYWORDS:  aviation  (civilian),  avionics,  data  link,  decision  aiding,  decision  making,  pilot 
aiding,  workload 


157.  Oliver,  J.G.  (1990).  improving  situational  awareness  through  use  of  intuitive 
pictorial  displays  (SAE  Technical  Paper  Series  No.  901829).  Warrendale,  PA: 
Society  of  Automotive  Engineers. 

This  paper  is  primarily  concerned  with  improving  SA  to  reduce  the  probability  of  errors  in  commercial 
airplanes.  It  adopts  a  definition  of  SA  from  the  Aerospace  Glossary  for  Human  Factors  Engineers 
(cited  on  p.  1),  "keeping  track  of  the  prioritized  significant  events  and  conditions  in  one's  environment." 
The  author  contends  that  the  loss  of  SA  in  the  airline  cockpit  is  a  recent  development  that  developed 
"in  direct  proportion  with  the  ever-dwindling  demand  for  the  crew  to  directly  control  the  flight 
situatiori"  (p.  2).  The  author  suggests  that  the  vast  majority  cf  attempts  to  improve  SA  have  consisted 
of  procedural  changes  to  force  the  crews  to  handle  more  information  in  the  hope  that  they  will 
understand  and  retain  it.  In  assessing  these  efforts,  the  author  states  (p.3):  "While  the  industry 
campaign  to  improve  situational  awareness  through  procedural  modification  has  been  partially 
succe.ssful,  it  can  never  be  totally  successful.  By  design,  procedures  force  the  crews  to  perform  a 
seemingly  endless  series  of  make-work  tasks  that  lead  to  boredom  and  complacency,  which,  in  turn, 
lead  to  a  loss,  not  a  gain,  of  situational  awareness."  As  an  alternative,  the  use  of  intuitive  pictorial 
displays  is  advocated  as  a  means  to  enhance  SA.  Several  display  systems  arc  discussed.  For  example, 
the  Navigation  display  is  cited  as  an  example  of  a  display  that  has  enhanced  SA,  while  the  Electronic 
Flight  Instruments  (EFIS)  display  is  cited  as  an  example  of  a  display  that  failed  to  enhance  SA.  The 
author  concludes  (p.  7):  "What  is  needed  if  we  really  are  interested  in  greater  situational  awareness,  is 
not  additional  Hp  service,  but  a  commitment  from  industry  to  provide  pictorial  displays  in  today's 
cockpits.  Until  we  do,  we  should  expect  the  incidence  of  accidents/incidents,  attributed  to  the  crew's 
loss  of  situational  awareness,  to  continue  essentially  unchanged." 

KEYWORDS:  aviation  (civilian),  avionics,  definition,  display  design,  pilot  aiding,  pilot 
error 
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158.  Parrish,  R.V.,  Busquets,  A.M.,  and  Williams,  S.P.  (1990).  Recent  research  results 
in  stereo  3-D  pictorial  displays  at  Langley  Research  Center.  In  Proceedings  of  the 
lEEE/AI A  A/NASA  9th  Digital  Avionics  Systems  Conference  (pp.  529-539).  New 
York:  Institute  of  Electrical  and  Electronics  Engineers. 

This  paper  summarizes  the  results  of  several  studies  conducted  at  NASA  -  Langley  Research  Center  on 
the  possible  application  of  stereoscopic  3-D  pictorial  displays.  Summarizing  the  implications  of  the 
studies,  the  authors  state  (p.  S29);  These  studies  demonstrated  increased  pilot/vehicle  performances 
through  situational  awareness  enhancements  of  pictorial  displays  with  stereopsis  cuing."  The 
improvement  of  S  A  was  generally  inferred  from  the  improvement  in  some  performance-based  metric, 
such  as  deviation  from  a  desired  path  or  stability  of  a  hover.  Although  it  is  stated  that  subjective 
reactions  supported  the  S  A  improvement  as  well,  no  explanation  of  what  subjective  measures  were 
collected  is  offered.  The  studies  were  conducted  in  a  variety  of  simulated  flight  tasks.  The  tasks 
studied  included  commercial  aviation  landing  patterns,  hovering  a  helicopter,  and  using  helmet-mounted 
displays  (HMDs).  The  technical  issues  surrounding  the  use  of  stereoscopic  displays  are  also  discussed. 

KEYWORDS:  aviation  (civilian),  display  design,  helicopter,  helmet-mounted  displays 
(HMDs),  measurement,  performance-based  measures 


159.  Perrott,  D.R.,  Sadralodabai,  T.,  Saberi,  K.,  and  Strybel,  T.Z.  (1991).  Aurally  aided 
visual  search  in  the  central  visual  field:  Effects  of  visual  load  and  visual 
enhancement  of  the  target.  Human  Factors,  33,  389-400. 

The  use  of  3-D  audio  to  enhance  performance  in  a  laboratory  visual  search  task  was  investigated.  It 
was  argued  that  3-D  audio  combined  with  a  head-coupled  display  system,  "could  improve  situational 
awareness  by  informing  that  critical  visual  infonnation  outside  the  cuaent  visual  field  is  available"  (p. 
390).  The  results  supported  this  suggestion.  Aurally  guided  search  improved  detection  of  targets  in 
the  visual  periphery,  especially  when  large  numbers  of  distractors  were  present. 

KEYWORDS:  3*^  audio  displays,  display  design,  measurement,  performance-based 
mea-sures,  target  identification 
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160.  Pettitt,  M.A.  (1993).  Cockpit  crises  and  decision  making:  Impiications  for  piiot 
training,  in  Proceedings  of  the  Seventh  International  Symposium  on  Aviation 
Psychology  (Voiume  1,  pp.  221-225).  Coiumbus:  The  Ohio  State  University. 

The  paper  explores  the  psychosocial  dynamics  of  flight  crews  facing  a  crisis  situation.  According  to  the 
author,  a  crises  is  characterized  by  a  change  in  the  environment  that  threatens  the  goals  of  the  team, 
generally  involves  substantial  risk,  and  generally  allows  little  time  for  decision  making.  Data  was 
collected  fi'om  a  survey  distributed  to  over  600  Los  Angeles-based  airline  pilots.  One  hundred 
eighty-five  of  the  surveys  were  returned.  The  survey  form  described  an  example  crises  situation  and 
then  asked  the  pilots  to  rate  the  crises  characteristics  of  the  .scenario  on  numerous  dimensions  using  a 
Likert-like  scale.  The  results  indicated  that  a  crucial  component  of  the  perception  of  crisis  was  the 
sense  of  urgency  associated  with  a  lack  of  time.  The  author  related  this  finding  to  SA  (p.  223),  "The 
results  of  this  study  suggest  that  it  would  be  beneficial  to  expand  the  concept  of  situational  awareness 
to  include  (a)  the  concept  of  crisis  and  (b)  the  accurate  assessment  of  decision  time  available  in  critical 
situations." 

KEYWORDS:  aviation  (civilian),  cognitive  processes,  decision  making,  definition, 
measurement,  questionnaire/survey  data,  subjective  measures,  team 
decision  making,  team  SA 


161.  Pew.  R.W.  (1994a).  Situation  awareness:  The  buzzword  of  the  '90s.  CSEPiAC 
Gateway,  5(1),  1-4. 

The  author  expresses  a  certain  skepticism  about  what  is  meant  by  the  term  SA  and  whether  the  concept 
really  adds  anything  beyond  the  usual  consideration  of  human  performance  requirements  in  a  system. 
Despite  reservations,  the  author  addresses  SA  because  it  has  become  (p.  2)  "the  buzzword  of  the  '90s" 
(having  replaced  "workload").  Starting  with  the  definitional  issue,  the  author  expresses  an  affinity  for 
Endsley's  (1988a)  definition  as  a  one  sentence  summary.  However,  it  is  suggested  that  definitions  of 
SA  should  begin  with  a  careful  consideration  for  the  definition  of  what  constitute  a  situation.  A 
working  definition  of  a  situation  is  offered  (p.  2):  "A  situation  is  a  set  of  environmental  conditions  and 
system  states  with  which  the  participant  is  interacting  that  can  be  characterized  uniquely  by  its  priority 
goals  and  response  options."  Given  a  situation,  the  SA  requirements  must  be  defined  to  produce  a 
standard  against  which  human  performance  can  be  measured.  It  is  proposed  that  the  standard  should 
be  described  at  three  levels:  (1)  an  abstract  ideal  (the  full  set  of  potentially  useful  information),  (2)  a 
physically  realizable  ideal  (a  subset  of  the  abstract  ideal  that  considers  the  constraints  imposed  by  a 
real  crew  station),  and  (3)  a  practically  realizable  ideal  (a  subset  of  the  physically  realizable  ideal  that 
considers  the  constraints  of  the  human  performance  capacities  and  limitations).  Training  and  selection 
are  suggested  as  ways  to  reach  the  physically  and  practically  realizable  ideals,  while  crew  station  design 
and  implementation  can  potentially  be  used  to  move  closer  to  the  abstract  ideal.  The  author  also 
differentiates  between  situation  awareness  as  a  product,  and  the  process  of  situation  assessment  which 
produces  the  awareness.  Finally,  the  author  suggests  research  strategies  involving  selection,  training, 
or  design  that  might  be  successful  in  determining  whether  SA  is  a  useful  concept  that  can  be 
differentiated  from  operational  human  performance. 

KEYWORDS:  definition,  display  design,  individual  differences,  measurement,  training 
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162.  Pew,  R. A.  (1994b).  An  introduction  to  the  concept  of  situation  awareness.  In 
R.D.  Gilson,  D.J.  Garland,  and  J.M.  Koonce  (Eds.),  Situational  awareness  in 
complex  systems:  Proceedings  of  a  CAHFA  Conference  (pp.  17-23).  Daytona 
Beach,  FL:  Embry-Riddle  Aeronautical  University  Press. 

This  paper  covers  much  of  the  same  ground  as  the  Pew  (1994a)  paper  regarding  definition  and  the  need 
for  standards  in  SA  assessment,  albeit  in  more  detail  on  some  issues  and  with  some  changes  in 
terminology.  The  author  includes  a  more  detailed  discussion  of  how  active  memory  processes  play  a 
vital  role  in  SA.  Also,  the  author  identifies  a  question  that  he  considers  critical  for  any  SA  researcher  to 
address  in  order  to  justify  committing  significant  research  effort  to  SA  (p.  22):  "This  question 
concerns  the  generality  of  the  concepts  associated  with  SA.  Either  (1)  there  are  design  principles  and 
general  individual  abilities  to  be  discovered  that  are  applicable  across  specific  devices  and  tasks,  or  (2) 
every  task  and  device  is  unique  and  it  only  makes  sense  to  select,  train  and  design  to  promote  the 
crew's  performance  with  specific  display  configurations  and  response  requirements."  In  assessing  liis 
own  belief  regarding  this  question,  the  author  states  (p.  23):  "For  the  moment,  I  come  down  in 
between.  I  believe  it  is  not  possible  to  define  SA  independently  of  an  understanding  of  the  SA 
requirements  of  context-specific  situations.  But  I  also  believe  that  there  may  be  skills  in  doing  this  that 
are  more  general  and  that  are  worthy  of  evaluation." 

KEYWORDS:  cognitive  processes,  definition,  display  design,  individual  differences, 
measurement,  training 


163.  Picart,  J.A.  (1991,  May).  Expert  warfighters  with  battlefield  vision.  Military 
Review,  71(5),  51-60. 

The  paper  does  not  specifically  discuss  S  A.  However,  the  concept  of  "battlefield  vision"  seems  to  be 
unmistakably  similar  to  SA.  The  US  Army  Field  Manual  22-103  defines  battlefield  vision  as,  "the 
ability  to  intuitively  sense  the  significance  of  battlefield  events  for  future  actions"  (cited  by  Picart,  p. 
51).  The  author  describes  the  relationship  of  battlefield  vision  to  pattern  recognition  and  expertise 
among  other  topics.  The  author  also  describes  the  "decision  support  template"  as  an  aid  for  battlefield 
decision  making  and  proposes  how  the  training  of  officers  should  be  organized  to  develop  commanders 
with  good  battlefield  vision. 

KEYWORDS:  battlefield  SA,  cognitive  processes,  decision  aiding,  decision  making, 
expertise,  training 
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164.  Pritchett,  A.,  and  Hansman,  R.J.  (1993).  Preliminary  analysis  of  pilot  ratings  of 
"party  line"  information  importance.  In  Proceedings  of  the  Seventh  International 
Symposium  on  Aviation  Psychology  (yo\un\e  1,  pp.  360-366).  Columbus:  The 
Ohio  State  University. 

Tfie  authors  hypothesized  that  data-linked  air  traffic  control  might  degrade  SA  in  the  commercial  airline 
cockpit.  This  hypothesis  was  based  on  the  loss  of  "party-line  information"  (PLI),  the  information  that 
any  pilot  can  pick  up  by  listening  to  air  traffic  control's  communications  with  other  aircraft.  Over 
4,000  surveys  were  sent  out  to  pilots  to  assess  their  opinions  on  this  topic.  Survey  forms  were 
distributed  to  a  wide  variety  of  pilot  types;  including  General  Aviation,  Commuter  Airline,  Major 
Airline,  Air  Force,  and  Navy  pilots.  At  the  time  of  the  writing  of  the  article  715  completed  surveys  had 
been  returned  and  used  in  the  analyses.  One  key  question  on  the  survey  was,  "What  does  the  'Big 
Picture'  mean  to  you?"  The  most  common  response  concerned  the  location  of  other  aircraft.  In  order, 
this  was  followed  by  knowledge  of  weather,  predicting  future  events,  and  communication. 

Summarizing  the  implications  of  their  results,  the  authors  conclude  (p.  363),  "traffic  and  weather 
information  should  be  given  high  priority  in  the  development  of 'Party  Line'  compensation  strategies." 

KEYWORDS:  air  traffic  control,  aviation  (civilian),  avionics,  data  link,  measurement,  pilot 
aiding,  questionnaire/survey  data,  subjective  measures 


165.  Rantanen,  E.  (1994).  The  role  of  dynamic  memory  in  air  traffic  controllers' 
situation  awareness.  In  R.D.  Gilson,  D.J.  Garland,  and  J.M.  Koonce  (Eds.), 
Situational  awareness  in  complex  systems:  Proceedings  of  a  CAHFA  Conference 
(pp.  209-215).  Daytona  Beach,  FL:  Embry-Riddle  Aeronautical  University  Press. 

Rantanen  equates  the  concept  of  S A  with  the  "picture"  that  air  traffic  controllers  must  maintain  to 
perform  their  tasks.  The  "picture"  is  defined  as  follows  (p.  209):  "The  air  traffic  controller's  picture  is 
a  mental  model  of  the  airspace  architecture,  layout  of  the  runways  at  airports,  rules  and  standard 
procedures  regulating  the  conduct  of  flights,  and  positions,  flight  data,  and  performance  characteristics 
of  the  aircraft  operating  within  this  system.  Also  included  in  the  picture  are  numerous  other  factors 
relevant  to  the  traffic  situation,  such  as  the  weather,  operational  status  of  navigation  aids  and  the  ATC 
equipment,  staffing,  and  sectorization  within  the  facility,  and  possible  irregularities  within  these." 

Within  this  framework,  Rantanen  discusses  the  role  of  dynamic  memory  processes  in  maintaining  the 
air  traffic  controller's  SA.  This  leads  to  the  conjecture  that  unless  the  application  of  automation  to  air 
traffic  control  is  carefully  guided  it  might  reduce  workload  by  eliminating  the  dynamic  memory 
processing  that  is  essential  to  good  SA. 

KEYWORDS:  attention,  air  traffic  control,  cognitive  processes,  definition,  mental  models, 
multiple  resource  theory,  workload 


123 


166.  Redding,  R.E.  (1992).  Analysis  of  operational  errors  and  workload  in  air  traffic 
control.  In  Proceedings  of  the  Human  Factors  Society  36th  Annual  Meeting 
(Volume  2,  pp.  1321-1325).  Santa  Monica,  CA:  Human  Factors  Society. 

The  paper  reviews  a  major  curriculum  redesign  effort  undertaken  to  improve  the  training  of  enroute  air 
traffic  controllers.  A  cognitive  task  analysis  of  current  en  route  controllers  suggested  that  failure  to 
maintain  SA  was  the  primary  cause  of  air  traffic  control  errors.  Failure  to  maintain  SA  was  linked  to 
inappropriate  mental  models  guiding  task  performance.  Proposals  for  the  modification  of  air  traffic 
control  training  to  improve  SA  are  discussed. 

KEYWORDS:  S'*'  traffic  control,  cognitive  processes,  expertise,  measurement,  mental 
models,  retrospective  measures,  subjective  measures,  training 


167.  Redding,  R.E.,  and  Cannon,  J.R.  (1992).  Expertise  in  air  traffic  control  (ATC): 
What  is  it,  and  how  can  we  train  for  it?  In  Proceedings  of  the  Human  Factors 
Society  36th  Annual  Meeting  (Volume  2,  pp.  1326-1330).  Santa  Monica,  CA: 
Human  Factors  Society. 

This  paper  presents  further  discussion  of  some  of  the  issues  raised  in  Redding  (1992).  In  this  paper, 
the  nature  of  expertise  within  the  air  traffic  control  domain  is  the  main  focus.  Maintaining  SA  is 
considered  a  key  component  of  expertise  in  air  traffic  control. 

KEYWORDS:  a'*'  traffic  control,  cognitive  processes,  expertise,  measurement,  mental 
models,  performance-based  measures,  questionnaire/survey  data, 
retrospective  measures,  subjective  measures,  training 
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168.  Regal,  D.M.,  Rogers,  W.H.,  and  Boucek,  G.P.  (1988).  Situational  awareness  in 
the  commercial  flight  deck:  Definition,  measurement,  and  enhancement  (SAE 
Technical  Paper  Series  No.  881508).  Warrendale,  PA:  Society  of  Automotive 
Engineers. 

This  paper  provides  a  description  and  breakdown  of  elements  involved  in  S  A  for  the  commercial 
flight  environment.  First,  the  authors  state  that  SA  can  be  defined  at  several  levels.  At  the  highest 
level,  they  believe  SA  "simply  means  that  the  pilot  has  an  integrated  understanding  of  factors  that  will 
contribute  to  the  safe  flying  of  the  aircraft  under  normal  or  non-normal  conditions"  (p.  1).  SA  is 
broken  dovm  into  components  of  spatial  awareness,  awareness  of  environment,  awareness  of  aircraft 
performance,  aircraft  systems  awareness,  and  crew  awareness,  with  examples  provided  for  each 
component.  Several  benefits  can  be  gained  from  enhancing  SA,  and  these  are  listed  and  discussed. 

They  are  improved  safety,  reduced  workload,  enhanced  pilot  performance,  an  expanded  range  of 
operation,  and  better  decision-making.  As  regards  measurement,  the  authors  state,  "We  believe  that  in 
order  for  [SA]  to  serve  as  a  flight  deck  design  criterion,  its  measurement  must  be  quantified, 
standardized,  and  validated"  (p.  3).  Direct  measurement  of  SA  via  asking  a  series  of  questions  is 
advocated,  along  with  measurement  of  pilot  performance  in  simulator  and  laboratory  settings.  Finally, 
enhancing  SA  through  better  flight  deck  design,  air  crew  coordination,  crew  training,  and  artificial 
intelligence  aids  is  recommended. 

KEYWORDS:  aviation  (civilian),  cockpit  resource  nnanagement  (CRM),  definition, 

measurement,  memory  probe  measures,  performance-based  measures,  pilot 
aiding,  workload 


169.  Rodgers,  M.D.,  and  Duke,  D. A.  (1994).  SATORI:  Situation  assessment  through 
the  re-creation  of  incidents.  In  R.D.  Gilson,  D.J.  Garland,  and  J.M.  Koonce  (Eds.), 
Situational  awareness  in  complex  systems:  Proceedings  of  a  CAHFA  Conference 
(pp.  217-225).  Daytona  Beach,  FL:  Embry-Riddle  Aeronautical  University  Press. 

This  article  discusses  the  development  of  the  Situation  Assessment  through  the  Re-Creation  of 
Incidents  (SATORI)  system.  The  SATORI  system  uses  the  data  that  is  already  recorded  within  the  air 
traffic  control  system  to  recreate  the  screen  displays  that  existed  during  any  specified  incident.  It  is 
intended  to  allow  close  study  of  and  operational  error  (0£)  incidents  that  occur.  Numerous  potential 
uses  for  the  SATORI  system  are  discussed,  including  the  use  as  a  research  tool  to  investigate  the 
effects  of  proposed  automation  on  air  traffic  controller  SA. 

KEYWORDS;  air  traffic  control,  training 
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170.  Rouse,  D.M.,  and  Hummel,  T.C.  (1987).  The  pilot's  associate:  Enhancing 

situational  awareness  through  cooperating  expert  systems  (SAE  Technical  Paper 
Series  No.  871896).  Warrendale,  PA:  Society  of  Automotive  Engineers. 

The  authors  discuss  the  design  of  the  Pilot's  Associate  (PA)  system.  PA  is  a  program,  funded  by  the 
Defense  Advanced  Research  Projects  Agency  (DARPA)  and  managed  Jointly  with  the  US  Air  Force 
Wright  Aeronautical  Laboratory.  PA  is  described  by  the  authors  as  (p.  1),  "a  network  of  cooperating 
expert  systems  designed  to  enhance  pilot  situational  awareness  by  assisting  the  pilot  with  information 
management  and  aircraft  control."  PA  uses  five  cooperating  subsystems  (i.e.,  h^ssion  Planner,  Tactics 
Planner,  Situation  Assessment,  System  Status,  and  Pilot-Vehicle  Interface)  and  an  executive  operating 
system.  The  goals  for  PA  include  managing  (but  not  necessarily  reducing)  pilot  workload,  converting 
raw  data  into  prioritized  information,  communicating  this  information  to  the  pilot,  and  advising  the 
pilot. 

KEYWORDS:  aviation  (military),  avionics,  decision  aiding,  pilot  aiding,  workload 


171.  Salas,  E.,  Stout,  R.J.,  and  Cannon-Bowers,  J.A.  (1994).  The  role  of  shared  mental 
models  in  developing  shared  situational  awareness.  In  R.D.  Gilson,  D.J.  Garland, 
and  J.M.  Koonce  (Eds.),  Situational  awareness  in  complex  systems:  Proceedings 
of  a  CAHFA  Conference  (pp.  297-304).  Daytona  Beach,  FL:  Embry-Riddle 
Aeronautical  University  Press. 

This  article  extends  the  SA  concept  from  the  more  typical  individual  performer  setting  to  team 
environment.  The  concepts  of  mental  models  and,  more  specifically,  shared  mental  models  are 
discussed.  After  reviewing  some  relevant  literature  the  authors  conclude  (p.  300):  "Summarizing  the 
results  of  both  literature  related  to  shared  mental  models  and  performance  and  team  situational 
awareness  and  performance,  the  following  conclusions  can  be  made:  1)  mental  models  are  important 
for  individual  situational  awareness;  2)  shared  mental  models,  which  allow  each  member  to  form 
adequate  explanations  and  expectations  of  task  and  team  actions,  are  important  to  team  performance; 
and  3)  team  situational  awareness  is  important  to  team  performance."  A  cognitive  model  is  then  used 
to  describe  how  team  SA  contributes  to  team  performance. 

KEYWORDS:  cognitive  processes,  mental  models,  team  SA 
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172.  Sarter,  N.B.  (Panel  Organizer),  and  Woods,  D.D.  (Panel  Chair)  (1990).  Situation 
awareness  in  the  advanced  commercial  aircraft  cockpit.  In  Proceedings  of  the 
Human  Factors  Society  34th  Annuai  Meeting  (Volume  1,  pp.  21-25).  Santa 
Monica,  CA:  Human  Factors  Society. 

This  set  of  abstracts  summarizes  a  panel  session.  This  panel  was  primarily  concerned  with  the  potential 
impact  of  automation  on  SA  and  the  lack  of  an  agreed-upon  definition  and  conceptual  foundation  for 
SA.  Six  abstracts  follow  the  introduction:  Rolf  Braune'(^M/owar/o/j  in  Commercial  Transport 
Airplanes:  The  Role  of  Situational  Awareness)-,  Everett  Palmer  (Crew  Situation  Awareness)-,  William 
Rogers  {What  Implications  does  Human-Centered  Automation  have  for  Situational  Awareness 
Requirements?)-,  Nadine  Sarter  {Situation  Awareness  -  A  Critical  but  Ill-Defined  Phenomenon)-, 
Christopher  Wickens,  Kelly  Harwood,  Anthony  Andre,  and  Anthony  Aretz  {Situation  Awareness, 

Visual  Momentum,  and  Frames  of  Reference)-,  and  Earl  Wiener  {Barriers  to  Situational  Awareness  in 
the  CocUpit  and  Possible  Intervention  Strategies). 

KEYWORDS:  aviation  (civilian),  avionics,  cockpit  resource  management  (CRM),  cognitive 
processes,  definition,  display  design,  pilot  aiding,  team  SA 


173.  Sarter,  N.B.,  and  Woods,  D.D.  (1991).  Situation  awareness:  A  critical  but 
ill-defined  phenomenon.  The  International  Journal  of  Aviation  Psychology,  1, 
45-57. 

The  article  is  very  concerned  with  adding  rigor  to  the  concept  of  situation  awareness.  The  considerable 
conceptual  overlap  between  situation  awareness  and  mental  models  is  noted,  but  a  distinction  is  drawn 
between  these  concepts.  The  authors  point  out  that  (p.  49),  "Mental  models  refer  to  systems  or 
devices  that  can  be  represented  by  a  finite  number  of  elements  and  algorithms,  whereas  situation 
awareness  is  about  a  continuously  changing  open  system  involving  a  large  variety  of  unpredictable  and 
interacting  agents  and  devices."  This  distinction  also  illustrates  the  authors'  concept  of  the  central  role 
played  by  temporal  factors  in  situation  awareness.  The  complexity  of  the  mental  models  involved  in 
situation  awareness  evolving  dynamically  over  time  lead  to  methodological  concerns  about  approaches 
that  have  been  tried  for  measuring  situation  awareness.  The  authors  fear  that  attempts  to  assess 
situation  awareness  via  specific  probes  for  information  may  be  "missing  the  phenomena"  of  overall 
situation  av/areness.  More  detailed  analysis  of  the  contents  of  memory,  such  as  that  acquired  through 
Endsley's  SAGAT,  are  considered  to  be  potentially  troublesome  because  of  their  intrusiveness  and 
because  the  fi'eezing  procedure  may  not  (p.54)  "provide  data  about  the  natural  character  and  occurrence 
of  situation  awareness."  Situation  awareness  measurement  procedures  that  rely  on  debriefing 
procedures  after  an  event  were  considered  to  be  flawed  because  they  will  not  allow  access  to 
unconscious  information  that  could  have  been  brought  into  awareness  if  the  appropriate  triggering 
conditions  were  present.  Instead  of  these  approaches  the  authors  suggest  that  investigation  of 
situation  awareness  "requires  the  staging  of  complex  dynamic  situations  and  the  development  of 
less-intrusive  in-flight  probing  techniques  to  assess  the  pilot's  ability  to  adequately,  and  in  the  most 
timely  manner,  receive  and  integrate  flight-related  information"  (p,  55). 

KEYWORDS:  cognitive  processes,  definition,  measurement,  memory  probe  measures, 
mental  models,  performance-based  measures,  SAGAT 
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174.  Sarter,  N.B.,  and  Woods,  D,D.  (1992).  Pilot  interaction  with  cockpit  automation: 
Operational  experiences  with  the  flight  management  system.  The  International 
Journal  of  Aviation  Psychology,  2,  303-321. 

The  authors  examined  the  impact  of  advanced  automation  on  situation  awareness  and  performance  in 
the  Boeing  B-737-300.  In  specific,  the  pilots'  interactions  with  the  flight-path  control  functions  of  the 
Flight  Management  System  (FMS)  were  studied.  Two  studies  were  conducted:  (1)  a  suivey  study  of 
B-737-300  pilots,  and  (2)  observations  of  pilots  undergoing  transition  training  in  a  simulator. 

Discussing  the  results  of  both  studies,  the  authors  report  (p.  3 16),  "The  corpus  of  reported  and 
observed  difficulties  indicates  that  pilots  can  lose  situation  awareness  (Sarter  &  Woods,  1991) 
concerning  FMS  status  and  behavior."  Further,  the  authors  point  out  that  individual  pilots  develop 
their  own  limited  and  idiosyncratic  strategies  for  dealing  with  the  FMS  and  these  different  strategies 
may  cause  problems  among  the  team  members  in  trying  to  maintain  their  SA.  The  authors  suggest  that 
the  training  of  pilots  could  be  improved  (p.  320),  "If  pilots  were  provided  with  an  overall  mental 
representation  of  the  functional  structure  of  the  FMS,  they  would  be  better  able  to  manage  and  utilize 
the  automated  systems  in  unusual  or  novel  situations." 

KEYWORDS:  aviation  (civilian),  avionics,  cognitive  processes,  measurement,  mental 

models,  questionnaire/survey  data,  subjective  measures,  team  SA,  training 


175.  Sarter,  N.B.,  and  Woods,  D.D.  (1994a).  Pilot  interaction  with  cockpit  automation 
II:  An  experimental  study  of  pilots'  model  and  awareness  of  the  flight  management 
system.  The  International  Journal  of  Aviation  Psychology.  4.  1-28. 

This  article  expands  on  the  authors'  analysis  of  the  Flight  Management  System  (FMS)  started  in  their 
1992  paper  by  incorporating  the  SA  measurement  procedures  discussed  in  their  1991  paper.  The 
authors  used  probe  questions  designed  to  assess  the  quality  of  20  airline  pilots'  mental  models  of  the 
operation  of  the  FMS.  These  probe  questions  were  presented  during  the  performance  of  a  simulated 
flight  in  a  B-737-300  part-task  trainer.  Based  on  pilot  performance  and  responses  to  the  probe 
questions,  the  authors  reported  that  (p.  21),  "most  of  the  difficulties  in  pilot-automation  interaction  are 
related  to  a  lack  of  mode  awareness  and  to  gaps  in  pilots'  mental  models  of  the  functional  structure  of 
the  automation."  In  general,  the  pilots  had  a  difficult  time  managing  the  automation  in  several 
simulated  non-normal  time-critical  events.  The  authors  suggest  that  this  lack  of  understanding  of  the 
operation  of  the  FMS  is  inconsistent  with  the  maintenance  of  effective  SA.  Causes  of  and  possible 
solutions  for  improving  the  pilots'  SA  in  this  type  of  environment  are  discussed. 

KEYWORDS'  aviation  (civilian),  avionics,  cognitive  processes,  measurement,  memory 
probe  measures,  mental  models 
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176.  Sarter,  N.B.  and  Woods,  D.D.  (1994b).  "How  in  the  world  did  I  ever  get  into  that 
mode?"  Mode  error  and  awareness  in  supervisory  control.  In  R.D.  Gilson,  D.J. 
Garland,  and  J.M.  Koonce  (Eds.),  Situational  awareness  in  complex  systems: 
Proceedings  of  a  CAHFA  Conference  (pp.  111*123).  Daytona  Beach,  FL: 
Embry*Riddle  Aeronautical  University  Press. 

The  authors  discuss  one  type  of  SA,  called  "mode  awareness,"  as  a  problem  being  experienced  with 
applications  of  advanced  automation  in  the  cockpits  of  airline  aircraft.  Mode  awareness  refers  to  the 
knowledge  of  an  operator  concerning  which  mode  a  complex  system  is  currently  in,  and  the 
implications  of  being  in  that  mode  on  system  behavior.  The  authors'  interpretation  of  mode  awareness 
makes  use  of  the  concept  of  mental  models,  especially  focusing  on  the  possible  causes  and  effects  of 
"buggy"  mental  models.  In  the  course  of  the  discussion,  the  authors  address  the  issue  of  a  definition 
for  SA  (p.  120);  "First,  extended  efforts  to  develop  the  'right'  definition  or  a  consensus  definition  of 
situation  (and  mode)  awareness  will  probably  not  be  constructive.  Rather,  the  term  ‘situation 
awareness'  should  be  viewed  as  just  a  label  for  a  variety  of  cognitive  proces.sing  activities  that  are 
critical  to  dynamic  event-driven  and  multi-task  fields  of  practice."  Among  the  cognitive  processing 
domains  suggested  as  related  to  SA  the  areas  of  attention  and  memory  processes  are  prominent.  The 
authors  also  address  the  issue  of  measuring  SA.  They  divide  the  potential  measures  of  SA  into  three 
main  areas;  subjective  ratings,  explicit  performance  measures,  and  implicit  performance  measures. 
Subjective  ratings  (such  as  SART)  and  explicit  performance  measures  (such  as  SAGAT)  are  assessed  as 
having  limited  utility  due  to  conceptual  and/or  practical  flaws.  Implicit  performance  measures,  while 
not  without  some  shortcomings,  are  presented  as  a  viable  approach  for  assessing  SA. 

KEYWORDS;  attention,  aviation  (civilian),  avionics,  cognitive  processes,  definition, 

implicit  measures,  measurement,  memory  probe  measures,  mental  models, 
subjective  measures,  SAGAT,  SART 


177.  Schwartz,  D.  (1989).  Training  for  situational  awareness.  In  Proceedings  of  the 

Fifth  International  Symposium  on  Aviation  Psychology  (Volume  1,  pp.  44-54). 

Columbus:  The  Ohio  State  University. 

This  article  is  oriented  towards  pilots  and  designers  of  pilot  training  programs.  Schwartz's  objective  is 
to  outline  the  "elements  which  contribute  to  taking  correct  action  in  the  cockpit."  He  presents  "the 
safety  v  indow"  as  the  time  period  and  airspace  surrounding  approach,  landing,  and  takeoff,  where 
accidents  are  most  likely  to  occur.  The  author  breaks  SA  down  into  five  contributing  factors 
(experience  and  training,  physical  flying  skills,  spatial  orientation,  health  and  attitude,  and  cockpit 
management),  and  talks  about  each  in  detail.  These  elements  should  all  be  included  in  any  SA  pilot 
training  program.  Schwartz  points  out  the  importance  of  identifying  links  in  "the  error  chain"  of  events 
as  an  essential  part  of  cockpit  management  training;  the  error  chain  is  the  sequence  of  events  which, 
unless  interrupted,  often  leads  to  accidents.  If  a  crewmember  can  identify  and  correct  a  situation  in 
progress  which  is  characterized  by  one  of  Schwartz's  1 1  signs  that  an  error  chain  is  underway,  an 
accident  may  be  averted.  Schwartz  recommends  all  flight  crew  training  programs  include  tliis  type  of 
information. 

KEYWORDS;  aviation  (civilian),  cockpit  resource  management  (CRM),  pilot  error,  training 
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178.  Schwartz,  D.  (1993,  May)  Training  for  situational  awareness.  Air  Line  Pilot,  6>2(4), 
20-23. 

This  appears  to  be  a  republication  of  the  Schwartz  (1989)  paper,  and  is  reviewed  under  that  listing. 
KEYWORDS:  aviation  (civilian),  cockpit  resource  management  (CRM),  pilot  error,  training 


179.  Scott,  W.B.  (1989,  April  17).  Falcon  Eye  Flir,  GEC  helmet  aid  F-16  mission 
flexibility.  Aviation  Week  &  Space  Technology,  130(6),  34-36,  41,  45-47. 

The  Falcon  Eye  system  is  an  infrared  sensor  mounted  in  front  of  an  F-16  cockpit.  The  sensor  is  placed 
in  a  ball-shaped  unit  that  can  rotate  to  aim  the  sensor  in  different  directions.  The  direction  of  the 
sensor  is  determined  by  a  system  that  detects  movements  of  the  pilot's  head.  The  sensor  points  in  the 
same  direction  as  the  pilot's  head.  Imagery  from  the  sensor  is  presented  by  a  helmet-mounted  display. 
The  author  argues  that  Falcon  Eye  "adds  an  excellent  degree  of  tactical  flexibility  and  night  situational 
awareness  by  allowing  the  pilot  to  look  in  any  direction"  (p.  35).  The  system  is  intended  to  increase 
the  potential  for  carrying  out  close  air  support  and  battlefield  interdiction  (CAS/BAl)  missions. 

KEYWORDS:  air-to-ground  mission,  aviation  (military),  avionics,  display  design,  flight 
test,  helmet-mounted  displays  (HMDs),  pilot  aiding,  target  identification 


180.  Scott,  W.B.  (1990,  April  30).  F-15E’s  night  attack  capability  assessed  in  low-level 
flight.  Aviation  Week  &  Space  Technology,  132(^6),  36-38,  43,  46. 

Pin  Aviation  Week  editor  flew  with  an  Air  Force  pilot  to  observe  the  use  of  the  F-15E  as  a  night  attack 
aircraft.  The  editor  flew  in  the  back-seat  weapons  systems  officer  (WSO)  station.  SA  was  mentioned 
in  a  discussion  of  one  of  the  displays  the  WSO  could  select  for  one  of  the  multifunction  CRT  displays. 
The  WSO  had  the  option  of  monitoring  the  head-up  display  (HUD)  symbology  and  the 
Forward-Looking  Infrared  (FLIR)  video.  The  editor  stated  that  (p.  43),  "I  found  that  a  periodic  quick 
glance  at  this  display  provided  some  of  the  basic  situational  awareness  cues  one  normally  would  get  by 
looking  outside  the  cockpit  during  a  day-time  flight." 

KEYWORDS:  air-to-ground  mission,  aviation  (miiitary),  display  design,  flight  test 


181.  Scott,  W.B.  (1991,  June  3).  777’s  flight  deck  reflects  strong  operations  influence. 
Aviation  Week  &  Space  Technology,  134(22),  52,  57-58. 

The  Boeing  777  cockpit  will  incorporate  fly-by-wire  controls  (a  first  for  Boeing),  new  color  liquid 
crystal  displays,  and  modem  avionics  (including  data-link  with  air  traffic  control).  While  putting 
together  these  elements,  the  Boeing  engineers'  first  objective  is  to  (p.  52),  "optimize  pilots'  situational 
awareness,  both  inside  and  outside  the  aircraft." 

KEYWORDS:  aviation  (civilian),  avionics,  data  link,  pilot  aiding 
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182.  Scott,  W.B.  (1992,  February  3).  F/A-18D  night  attack  features  boost  Marine  air 
effectiveness.  Aviation  Week  &  Space  Technology,  i36{5],  38-42. 

An  Aviation  Week  editor  flew  with  a  Marine  pilot  to  observe  how  the  Marines  were  using  night-vision 
goggles  (NVGs)  and  Forward-Looking  Infrared  (FLIR)  sensors  to  use  the  F-18D  as  a  night  attack 
aircraft.  SA  was  mentioned  in  interpreting  the  effect  of  the  NVGs  limited  field-of-view  on  low  altitude 
flying  (p.  41),  "Human  factors  research  has  detennined  that  low-altitude  situational  awareness  depends 
a  great  deal  on  visual  cues  from  peripheral  vision."  These  peripheral  vision  cues  are  missing  from  the 
NVGs.  To  compensate  for  the  lack  of  peripheral  vision,  the  training  of  the  Marine  F-18D  pilots 
emphasizes  that  pilots  must  keep  their  heads  moving  and  not  fixate  on  one  thing  in  the  outside  world. 

KEYWORDS:  air-to-ground  mission,  aviation  (military),  display  design,  flight  test,  pilot 
aiding,  training 


183.  Scott,  W.B.  (1992,  February  10).  New  F/A-18C/D  .software  to  integrate  sensor 
inputs,  reduce  pilot  workload.  Aviation  Week  &  Space  Technology,  136(Q),  46-47. 

The  Multi-Source  Integration  (MSI)  software  package  is  being  tested  for  possible  use  in  upgrading  the 
F-18  aircraft  avionics  for  air-to-air  combat.  There  are  two  main  components  to  the  software:  (1)  new 
mission  computer  algorithms  to  merge  data  from  several  onboard  sensors  to  present  a  single  "trackfile" 
for  each  target,  and  (2)  four  new  display  formats  to  present  the  information  (including  a  "situation 
awareness  format").  The  outcome  of  these  upgrades  is  described  by  Lt.  Cdr.  Jeff  Crutchfield,  chief  test 
pilot,  as  (p.  46),  "The  pilot  gets  improved  detection,  track  continuity,  better  [weapon]  accuracy,  and 
improved  situational  awareness." 

KEYWORDS:  air-to-air  mission,  aviation  (military),  avionics,  data  fusion,  data  link,  display 
design,  pilot  aiding 


184.  Scott,  W.B.  (1992,  March  23).  Douglas  uses  MD-11  cockpit  success  as 

springboard  for  designing  MD-12.  Aviation  Week  &  Space  Technology,  136{'\2), 
58-59. 

The  article  describes  the  philosophy  and  process  used  by  McDonnell  Douglas  to  guide  the 
implementation  of  automation  in  the  MD-12  aircraft.  The  overall  philosophy  is  described  as  "keeping 
the  pilot  in  the  loop."  As  automatic  systems  sense  new  conditions  and  adjust  for  them,  the  pilot  is 
notified  of  these  actions.  It  is  stated  that  (p.  59),  "Douglas  engineers  said  that  this  approach  ensures 
pilot  situational  awareness  and  the  ability  to  always  override  automatic  functions."  Effort  is  also  taken 
to  ensure  that  visual,  aural,  and  tactile  cues  of  what  is  happening  are  maintained.  However,  there  was 
still  some  concern  that  the  operation  of  the  aircraft  required  too  much  "head-down"  time.  Flight  crews 
have  argued  that  more  functions  should  be  implemented  through  the  glareshield  panel  display. 

KEYWORDS:  aviation  (civilian),  avionics,  display  design,  pilot  aiding 
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185,  Scott,  W.B.  (1993,  June  7).  Second-generation  Flirs  enhance  night  attack 
systems.  Aviation  Week  &  Space  Technology,  138{23),  143-145. 

The  article  reviews  the  Falcon  Knight  system.  The  Falcon  Knight  system  uses  two  head-steered 
Forward-  Looking  Infrared  (Flir)  sensors  that  present  outside  imagery  on  helmet-mounted  displays. 

The  Flir  information  is  integrated  with  the  on-board  fire-control  radar.  This  system  was  contrasted  to 
night  attack  systems  based  on  external  pod-carried  units.  One  of  the  main  advantages  of  the  Falcon 
Knight  system  was  felt  to  be  improved  SA,  especially  during  attacks.  For  example,  Falcon  Knight 
enabled  pilots  to  launch  Maverick  Missiles  two  seconds  sooner  than  they  would  in  an  unequipped  F-16. 

KEYWORDS:  air-to-ground  mission,  aviation  (military),  avionics,  data  fusion,  display 
design,  flight  test,  helmet-mounted  displays  (HMDs),  measurement, 
performance-based  measures 


186.  Secrist,  G.E.,  and  Hartman,  B.O.  (1993,  May).  States  of  awareness  I:  Subliminal 
perception  relationship  to  situational  awareness  (Tech.  Report  No. 

AL-TR-1 992-0085).  Brooks  Air  Force  Base,  TX:  Armstrong  Laboratory. 
(ADA267249) 

As  part  of  a  search  for  a  conceptual  foundation  for  SA,  the  authors  examined  many  areas  of  the 
behavioral  sciences.  Among  the  areas  that  seemed  to  have  a  relevant  relationship  to  SA  was  the  topic 
of  subliminal  perception.  As  described  by  the  authors  (p.  1),  "The  concept  of  subliminal  perception 
concerns  the  notion  that  individuals  can  acquire,  analyze,  process,  and  be  affected  by  sensory  stimuli 
below  the  level  of  conscious  awareness."  The  bulk  of  the  report  is  a  detailed  review  of  the  theoretical 
concepts  associated  with  subliminal  perception,  the  empirical  data  used  to  support  the  notion  of 
subliminal  perception,  and  the  critiques  of  the  notion.  Based  on  this  review,  the  authors  conclude  that 
"subliminal  perception  and  related  subconscious  information  acquisition  and  processing  capabilities  are 
an  untapped  reservoir  of  human  potential"  (p.  46),  The  authors  argue  that  subliminal  perceptual 
capabilities  could  potentially  be  developed  in  individuals  and  discuss  how  such  development  would 
enhance  SA  and  performance. 

KEYWORDS:  cognitive  processes,  review  paper 
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187.  Secrist,  G.E.,  and  Hartman,  B.O.  (1993).  Situational  awareness:  The  trainabiiity  of 
the  near-threshold  information  acquisition  dimension.  Aviation,  Space,  and 
Environmental  Medicine,  64,  885-392. 

Following  up  on  the  notions  presented  in  their  Hartman  and  Secrist  (1991)  paper,  the  authors  present 
the  results  of  an  experiment  to  examine  the  trainabiiity  of  near-threshold  information  processing.  The 
experiment  used  three  different  tasks  that  involved  the  detection,  recognition,  or  identification  of 
symbols  that  were  presented  for  duration  of  either  33  msec  or  67  msec.  Results  showed  that 
performance  on  these  tasks  improved  with  practice,  and  the  improvement  was  much  quicker  with  the 
67  msec  presentations.  There  was  no  test  of  whether  this  training  transferred  to  another  task 
environment,  however.  As  the  authors  themselves  state  (p.  891),  "We  have  demonstrated  only  that  a 
training  technique  derived  from  our  conceptual  structure  for  situational  awareness  has  shown  training 
effects  and  that  such  effects  are  internally  consistent." 

KEYWORDS:  cognitive  processes,  measurement,  performance-based  measures,  training 


188.  Selcon,  S.J.  and  Taylor,  R.M.  (1990,  April},  Evaluation  of  the  situational 
awareness  rating  technique  (SART)  as  a  tool  for  aircrew  systems  design,  in 
AGARD-CP-478,  Situational  Awareness  in  Aerospace  Operations  (pp.  5-1  to  5-8). 
Neuilly  Sur  Seine,  France:  Advisory  Group  for  Aerospace  Research  & 
Development.  (AD-A223939) 

The  Situational  Awareness  Rating  Technique  (SART)  was  developed  by  Taylor  as  a  means  of 
evaluating  aircrew  systems  designs.  The  ten  dimensional  SART  scale  consists  of  three  domains 
(Demand  on  Attentional  Resources,  Supply  of  Attentional  Resources,  Understanding)  covering  ten 
constructs  (such  as  Variability  of  Situation  under  "Demand,"  Concentration  under  "Supply,"  and 
Information  Quantity  under  "Understanding").  This  paper  discusses  the  three  experiments  conducted 
to  evaluate  the  SART.  Study  I  was  a  multiple  task  compatibility  study,  investigating  left/right 
hemisphere-specific  tasks  in  competition  for  attentional  resources.  Study  II  investigated 
attention-switching  capabilities  when  subjects  were  presented  with  different  attitude  references  during 
emergency  attitude  recovery.  Study  III  focused  on  the  reaction  times  of  subjects  when  presented  with 
visual,  auditory  or  both  type  warning  icons  describing  "Warning"  or  "Caution"  situations.  The  general 
discussion  provides  support  for  validating  the  SART  methodology. 

KEYWORDS:  attention,  attitude  displays,  cognitive  processes,  display  design, 

measurement,  performance-based  measures,  SART,  subjective  measures 


189.  Selcon,  S.J.,  and  Taylor,  R.M.  (1991).  Decision  support  and  situational 

awareness.  In  Designing  for  Everyone:  Proceedings  of  the  1 1th  Congress  of  the 
International  Ergonomics  Association  (Volume  1,  pp.  792-794).  Paris,  France: 
Taylor  &  Francis. 

This  paper,  along  with  4  other  SA  related  papers  presented  at  the  Congress,  were  reprinted  in  Taylor 
(1991)  and  is  reviewed  under  that  listing. 

KEYWORDS:  cognitive  processes,  decision  aiding,  decision  making,  pilot  aiding 
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190.  Selcon,  SJ.,  Taylor,  R.M.,  and  Korltsas,  E.  (1991).  Workload  or  situational 
awareness?:  TLX  vs.  SART  for  aerospace  systerrt  design  evaluation.  In 
Proceedings  of  the  Human  Factors  Society's  35th  Annual  Meeting  (Volume  1,  pp. 
62*66).  Santa  Monica,  CA:  Human  Factors  Society. 

This  experiment  contrasted  subjective  situation  awareness  ratings  (10-D  and  3-D  SART)  to  subjective 
workload  ratings  (NASA-TLX).  Twelve  RAF  pilots  participated  by  watching  videotaped  simulations 
of  air  combat.  The  videotaped  missions  were  previously  ranked  to  represent  differing  levels  of  task 
difficulty/attentional  demands.  Both  types  of  ratings  were  sensitive  to  changes  in  task  demands. 
However,  SAF.T  was  sensitive  to  experience  differences  that  NASA-TLX  failed  to  detect.  Overall,  the 
results  suggest  overlap  in  the  subjective  experiences  of  situation  awareness  and  mental  workload. 

KEYWORDS:  attention,  cognitive  processes,  measurement,  SART,  subjective  measures, 
workload 


191.  Selcon,  S.J.,  Taylor,  R.M.,  and  Shadrake,  R.A.  (1991,  June).  Information 
processing  advantages  of  multiple  sources  of  cockpit  information  (Institute  of 
Aviation  Medicine  Report  No.  672).  Farnborough,  UK:  RAF  Institute  of  Aviation 
Medicine. 

This  document  describes  three  studies  which  evaluated  the  processing  of  information  presented  by 
means  of  redundant  sources.  The  purpose  of  these  studies  of  "redundancy  gain"  was  to  explore  the 
comprehension  benefits  in  the  theoretical  framework  as  well  as  the  context  of  aircrew  system  design. 
The  first  experiment  used  a  bi-modal  (visual  and  verbal)  presentation  of  numbers  to  evaluate 
cross-modal  integrality.  The  second  used  colors,  words,  and  a  combination  of  the  two  to  determine 
redundancy  gains  at  the  "level  of  comprehension."  The  last  experiment  investigated  performance  gains 
related  to  waming/caution  icons  and  verbal  messages;  the  Situational  Awareness  Rating  Technique 
(SART)  was  used  to  measure  the  depth  of  understanding. 

KEYWORDS:  cognitive  processes,  display  design,  measurement,  performance-based 
measures,  pilot  aiding,  SART,  subjective  measures 


192.  Selcon,  S.J.,  Taylor,  R.M.,  and  Shadrake,  R.A.  (1992).  Multi-modal  cockpit 
warnings:  Pictures,  words,  or  both?  In  Proceedings  of  the  Human  Factors 
Society  36th  Annual  Meeting  (Volume  1,  pp.  57-61).  Santa  Monica,  CA:  Human 
Factors  Society. 

The  authors  examined  the  potential  benefits  of  linking  meaningful  pictorial  icons  and  auditory  signals 
together  in  multi-modal  warnings.  The  results  indicated  that  the  combination  of  signals  was  responded 
to  more  quickly  than  unimodal  signals.  The  3-Dimensionai  Subjective  Assessment  Rating  Technique 
(SART)  was  also  used  to  assess  the  effects  of  the  multimodal  signals.  The  subjects  ratings  suggested 
that  there  was  both  a  reduction  of  workload  and  an  increase  in  understanding  when  the  multimodal 
signals  were  used.  The  authors  also  proposed  a  formula  for  combining  the  three  raw  SART  scale  values 
into  a  single  overall  calculated  SA  rating.  This  formula  consisted  of  subtracting  the  Supply  rating  from 
the  Demand  rating,  and  then  subtracting  the  difference  from  the  understanding  rating 
[SA(c)“Under8tanding-(Demand-Supply)].  The  SA(c)  score  was  also  sensitive  to  the  benefits  of 
multimodal  warning  signals. 

KEYWORDS:  cognitive  processes,  display  design,  measurement,  performance-based 
measures,  pilot  aiding,  SART,  subjective  measures,  workload 


193,  Shaw,  R.L.  (1985).  Fighter  combat:  Tactics  and  maneuvering.  Annapolis,  MD; 
Naval  Institute  Press. 

During  the  course  of  discussing  air  combat,  SA  is  mentioned  repeatedly  as  one  contributor  to  success. 
In  fact,  the  trend  towards  adding  electronic  aids  to  the  cockpit  that  started  in  World  War  II  and 
continues  unabated  is  interpreted  within  a  SA  framework  (p.  291),  "All  of  this  electronic  technology  is 
intended  to  increase  the  situational  awareness  of  friendly  forces  in  battle  and  to  deny  such  valuable 
information  to  the  enemy.  The  incredible  attention  electronics  technology  has  received  in  recent  years 
is  a  clear  indication  of  the  importance  of  situational  awareness  in  air  combat."  Not  surprisingly,  SA  is 
expected  to  be  harder  to  maintain  as  the  number  of  aircraft  involved  in  combat  increases,  regardless  of 
whether  the  aircraft  are  friendly  or  enemy.  However,  SA  (actually  "degree  of  confidence  in  situation 
awareness")  is  described  as  one  of  the  vital  components  of  a  tactical  situation  that  should  be  considered 
during  the  selection  of  tactics  (pp.  367-368).  Also,  some  tactics  are  discussed  that  are  designed  to 
primarily  degrade  an  enemy's  SA  (pp.  370-371).  Overall,  the  book  shows  the  vital  role  that  SA  is 
thought  to  play  in  successfiil  air-to-air  combat. 

KEYWORDS:  air-to-air  mission,  aviation  (military),  avionics,  pilot  aiding,  training 
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194.  Smallwood,  W.L.  (1993).  Warthog:  Flying  the  A-10  In  the  Gulf  War. 

Washington.  DC;  Brassey's. 

1'his  book  reviews  the  role  played  by  A*  10  squadrons  in  the  1991  Gulf  War.  Although  used  in  several 
other  missions,  the  A- 10  was  de.signed  to  be  a  close^air  support  aircraft.  SA  is  only  mentioned  once 
(pp.  183-184),  "I  entered  the  kill  box  at  17,000  feet.  I  took  the  briefing  at  17,000  feet  and  in  the  course 
of  forty  minutes  of  flying  around  I  was  down  to  8,000  to  9,000  feet.  As  I  went  lower  my  SA 
[situational  awareness]  should  have  had  a  corresponding  increase,  but  It  didn't."  Shortly  after  this,  the 
aircraft  was  probably  hit  by  a  surface-to-air  missile  (SAM).  In  any  case,  the  aircraft  was  hit  and  the 
pilot  was  forced  to  eject.  The  pilot  was  captured  by  the  Iraqis. 

KEYWORDS:  air-to-ground  mission,  aviation  (military),  flight  test 


195.  Smith,  K.,  and  Hancock,  P.A.  (1994).  Situation  awareness  is  adaptive, 

externally-directed  consciousness.  In  R.D.  Gilson,  D.J.  Garland,  and  J.M  Koonce 
(Eds.),  Situational  awareness  in  complex  systems:  Proceedings  of  a  CAHFA 
Conference  (pp.  59-68).  Daytona  Beach,  FL:  Embry-Riddle  Aeronautical  University 
Press. 

The  authors  address  the  issue  of  a  definition  for  SA  (p.  60):  "We  extend  the  account  of  Tenney, 

Adams,  Pew,  Huggins,  and  Rodgers  (1992)  to  propose  that:  (i)  SA  is  best  defined  as  adaptive, 
externally-directed  consciousness;  and  (ii)  S  A  is  the  invariant  at  the  core  of  an  adapted  agent's  Neisser 
cycle  that  generates  both  up-to-the-minute  knowledge  and  action  that  anticipates  signals  in  the  task 
environment,"  The  remainder  of  the  paper  discusses  the  ramifications  of  this  viewpoint  of  SA.  The 
discussion  is  conducted  within  the  framework  of  the  ecological  psychology  movement.  Representation 
of  the  SA  invariant  is  demonstrated  with  an  example  from  air  traffic  control. 

KEYWORDS:  air  traffic  control,  cognitive  processes,  definition 


196.  Smyth,  C.C.,  Malkin,  F.J.,  and  DeBellis,  W.B.  (1990).  Design  considerations  for  a 
counterair  situation  awareness  display  for  Army  aviation.  In  American  Helicopter 
Society  46th  Annual  Forum  Proceedings  (Volume  II,  pp.  921-930).  Alexandria, 

VA:  American  Helicopter  Society. 

This  article  discusses  the  type  of  display  needed  to  equip  Army  helicopters  to  be  viable  in  an  air-to-  air 
combat  role  against  enemy  helicopters.  The  article  assumes  that  situation  information  would  be 
available  to  the  helicopter  through  data  links  to  off-board  sensor  systems.  This  infonnation  would  then 
be  used  to  guide  tactics  for  a  successful  engagement.  The  information  would  be  presented  to  the  pilot 
on  a  "situational  awareness  digital  map  display”  (p.  922).  The  information  that  needs  to  be  presented 
on  such  a  display,  approaches  to  formatting  the  information,  and  the  limits  of  current  helicopter  display 
technology  are  discussed. 

KEYWORDS;  air-to-air  mission,  avionics,  display  design,  data  fusion,  data  link,  helicopter, 
pilot  aiding 
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197.  Spick,  M.  (1988).  The  ace  factor:  Air  combat  &  the  role  of  situational  awareness. 
Annapolis:  Naval  Institute  Press. 

This  book  is  apparently  intended  for  a  mainstream  audience.  Much  of  the  book  is  concerned  with  the 
question  of  what  separates  expert  ace  pilots  from  the  remaining  majority  of  normal  military  pilots.  The 
author  argues  that  the  "ace  factor"  is  better  SA  abilities.  This  is  supported  by  a  review  of  the  liistory  of 
air  combat.  The  potential  for  improving  SA  in  future  air  combat  pilots  is  also  discussed,  generally 
within  the  framework  of  technological  advancements  in  avionics  and  cockpit  design. 

KEYWORDS:  air-to-air  mission,  aviation  (military),  avionics,  cognitive  measures, 
expertise,  pilot  aiding,  training 


198.  Stein,  K.J.  (1986,  December  1).  Complementary  displavs  could  provide 
panoramic,  detailed  battle  area  views.  Aviation  Week  &  Space  Technology, 
125{22),  111,  113,  115. 

The  article  defines  SA  as  (p.  113),  "the  knowledge  available  to  the  pilot  on  critical  matters  such  as  the 
overall  tactical  situation,  his  own  mission  profile,  weapons  status,  the  positions  and  objective  of 
friendly  aircraft,  disposition  and  apparent  objectives  of  hostile  flights,  presence  of  threats,  refueling 
rendezvous  and  other  mission  data."  Eugene  Adam  of  McDonnell  Aircraft  is  quoted  as  suggesting  (p. 

1 1 1),  "Pilots  are  not  so  much  overworked  as  they  are  underinformed."  The  article  discusses  two 
display  concepts  that  could  be  combined  to  improve  pilot  SA:  (1)  a  "Big  Picture"  cockpit  display  of 
about  300  square  inches  to  provide  an  overview  of  the  battle  situation,  and  (2}i  a  helmet-mounted 
HUD-like  display  for  targeting  weapon  systems  and  detailed  tactical  capabilities.  It  is  suggested  that 
these  display  innovations  could  help  to  avoid  the  pervasive  lack  of  S  A  that  has  resulted  in  two-thirds  of 
the  combat  pilots  being  shot  down  by  people  they  never  saw. 

KEYWORDS:  aviation  (military),  big  picture  displays,  definition,  display  design, 
helmet-mounted  displays  (HMDs),  pilot  aiding 
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201.  Stiffler,  D.R.  (1987,  April).  Exploiting  situational  awareness  beyond  visual  range 
(Student  Report  No.  87-2370).  Maxwell  Air  Force  Base,  AL:  Air  Command  and 
Staff  College.  (AD-B1 10786) 

The  author  discusses  issues  associated  with  a  definition  of  situation  awareness  for  air  combat. 

Numerous  issues  are  discussed  and  several  definitions  are  presented,  but  the  autitor  appears  to  settle  on 
defining  situation  awareness  as  (p.  13),  "the  ability  to  envision  the  current  and  near-term  disposition  of 
both  friendly  and  enemy  forces."  Situation  awareness  was  related  to  the  0-0-D-A  loop  concept  of 
combat  decision  making.  The  0-0-D-A  loop  consists  of  cycling  through  the  stages  of  Observe  - 
Orient  -  Decide  -  Act.  The  orientation  phase  is  emphasized  as  the  most  crucial  to  situation  awareness. 
During  orientation,  the  information  from  the  observation  stage  is  combined  with  knowledge  to  produce 
a  mental  image  of  the  situation.  In  regard  to  situation  awareness,  two  characteristics  are  emphasized  by 
the  author:  measurability  and  exclusivity.  Measurability  is  considered  to  be  "remarkably  easy;  one 
only  has  to  ask  the  pilot  after  the  mission  to  evaluate  his  own  SA"  (p.  6).  In  practice,  this  approach 
requires  a  debriefing  of  the  participants  in  a  mission  explaining  the  evolution  of  their  mental  pictures  as 
they  progressed  through  the  mission.  These  mental  pictures  are  compared  to  either  a  consensus  view 
of  other  participating  or  observing  pilots  or  to  a  playback  of  some  record  of  the  mission  or  both.  The 
author  cites  a  study  that  found  a  high  correlation  between  pilots'  self  rating  of  situation  awareness  and 
the  assessments  of  observing  pilots  as  evidence  of  the  validity  of  this  approach.  The  author  also 
cautions  against  trying  to  measure  situation  awareness  at  any  one  point  in  time  because  it  can  change 
rapidly  throughout  the  mission.  The  characteristic  of  exclusivity  of  situation  awareness  suggests  that 
in  air-to-air  combat  only  one  side  will  tend  to  have  good  situation  awareness  at  any  point  in  time.  This 
was  not  presented  as  a  theoretical  certainty,  but  as  an  empirical  observation.  As  one  side  gains 
situation  awareness  in  simulated  air-to-air  combat,  they  would  tend  to  act  in  such  a  way  as  to  disnjpt 
the  other  side's  0-0-D-A  loop  and  thus  diminish  their  situation  awareness.  Overall,  the  paper  is 
notable  for  its  strong  ties  to  the  operationally  relevant  aspects  of  situation  awareness  and  its  attempts 
to  connect  the  concept  of  situation  awareness  to  concepts  of  combat  decision  making. 

KEYWORDS:  aviation  (military),  cognitive  processes,  decision  making,  definition, 

measurement,  mental  models,  retrospective  measures,  subjective  measures 


202.  Stiffler,  D.R.  (1988,  Summer).  Graduate  level  situational  awareness.  USAF 
Fighter  Weapons  Review,  56(2),  15-17,  20. 

This  article  appears  to  be  a  summary  of  the  Stiffler  (1987)  article.  It  presents  the  same  main  ideas  and 
ties  them  even  more  directly  to  air  combat  tactics. 

KEYWORDS:  aviation  (military) 
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203.  Stiles,  G.J.,  and  Pearson,  R.P.  (1986,  December).  Designing  for  situational 
awareness.  Defense  Electronics,  fS(12),  84-86,  88. 

Ihe  authors  define  SA  as  (p,  85),  "a  combatant's  knowledge  of  the  battlefield  situation  about  him." 
They  suggest  that  the  SA  concept  can  be  usefully  subdivided  into  SA  about  different  aspects  of  the 
situation  (such  as,  weapons'  SA,  flight  performance  SA,  etc.).  Within  this  framework  they  analyze  the 
failure  for  some  electronic  warfare  (EW)  systems  to  gain  favor  within  the  cockpit  environment.  In 
many  cases,  the  inclusion  of  new  EW  systems  have  been  resisted  by  pilots  and  existing  EW  systems 
have  deactivated  by  pilots  in  combat  environments.  The  authors  argue  that  the  presentation  of  the  EW 
system  information  has  often  not  been  compatible  with  the  construction  and  maintenance  of  overall 
SA.  They  suggest  that  the  displays  must  be  better  designed  to  aid  SA.  Among  other  suggestions,  the 
authors  argue  that  a  good  display  should  assist  the  pilot  in  visually  locating  any  threats  outside  the 
cockpit  and  must  provide  easily  understood  and  accurate  range  information. 

KEYWORDS;  aviation  (military),  definition,  display  design,  pilot  aiding 


204.  Stratton,  M.D.,  Wilson,  G.F.,  and  Crabtree,  M.S.  (1993).  An  analysis  of 

physiological  metrics  in  the  study  of  pilot  situational  awareness.  In  Proceedings 
of  the  Seventh  International  Symposium  on  Aviation  Psychology  (Volume  2,  pp. 
901-905).  Columbus:  The  Ohio  State  University. 

The  authors  approached  this  study  with  the  goal  of  determining  whether  physiological  measurement 
techniques  which  are  sensitive  to  mental  workload  can  be  used  to  indirectly  measure  SA.  EEG,  heart 
rate  (interbeat  interval),  and  eye  blink  (duration  and  rate)  measures  were  used.  Subjects  were  12 
college  students.  They  performed  a  simulated  flying  task  in  a  Simulator  for  Tactical  Operations 
Research  and  Measurement  (STORM)  system.  The  out-the-window  display  varied  between  two 
"daylight"  conditions  (one  with  red  and  one  with  brown  tanks),  and  a  global  satellite  positioning  (GPS) 
display.  For  half  the  trials,  an  audio  signal  warned  subjects  of  a  nearby  surface-to-air  missile  (SAM), 
which  indirectly  indicated  proximity  to  ground  targets  (tanks).  The  most  difficult  conditions  showed 
an  increase  in  power  in  the  theta  band  relative  to  other  bands.  Therefore,  the  authors  conclude  that 
using  EEG  as  an  indirect  measure  of  SA  shows  promise.  The  heart  rate  and  eye  blirdc  measures  were 
not  sensitive  to  display  conditions. 

KEYWORDS:  combat  simulation,  measurement,  physiological  measures,  workload 
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205.  Sullivan,  C.,  and  Blackman,  H.S.  (1991).  Insights  into  pilot  situation  awareness 
using  verbal  protocol  analysis.  In  Proceedings  of  the  Human  Factors  Society  35th 
Annual  Meeting  (Volume  1,  pp.  57-61).  Santa  Monica,  CA:  Human  Factors 
Society. 

Tl)e  authors  related  concepts  concerning  the  relative  roles  of  long-term  and  short-term  memory  in 
creating  and  maintaining  situation  awareness  to  techniques  for  collecting  and  analyzing  verbal  protocol 
information.  Their  hypothesis  was  stated  as  follows  (p.  57),  "Given  that  experienced  pilots  are  more 
likely  to  have  long-term  memory  structures  that  alleviate  the  load  on  working  memory,  it  is  expected 
that  expert  pilots  will  require  less  information  from  the  intercept  environment  to  maintain  SA.  Thus 
expert  pilots  will  have  fewer  verbalizations  regarding  the  updating  and  maintenance  of  intercept 
information."  This  hypothesis  was  tested  by  collecting  concurrent  verbal  protocols  from  six  expert  and 
six  novice  F-16  pilots  while  they  performed  a  variety  of  simulated  air-to-air  intercepts.  Contrary  to  the 
hypothesis,  no  main  effect  of  pilot  experience  was  detected  in  the  verbal  protocol  analysis.  However, 
there  was  an  interaction  between  pilot  experience  and  number  of  enemy  aircraft.  Novices  spoke  much 
more  than  experts  when  confronted  with  two  opponents,  but  verbalized  about  the  same  as  experts 
when  confronted  with  either  one  or  four  enemies.  The  results  from  this  study  can  not  be  considered 
conclusive,  but  nevertheless  suggest  that  verbal  protocol  analysis  might  be  a  promising  avenue  for 
future  research. 

KEYWORDS:  expertise,  cognitive  processes,  combat  simulation,  measurement,  subjective 
measures,  verbal  protocols 


206.  Tarrel,  R.J.  (1985).  Human  factors  associated  with  runway  transgressions.  In 
Proceedings  of  the  Third  Symposium  on  Aviation  Psychology  (pp.  713-720). 
Columbus:  The  Ohio  State  University. 

This  article  reported  on  the  results  of  an  analysis  of  runway  transgression  errors  by  commercial  pilots 
that  have  been  reported  to  NASA's  Aviation  Safety  Reporting  System  (ASRS).  A  total  of  104  ASRS 
runway  transgression  reports  were  analyzed.  Each  report  was  classified  according  to  whether  it 
involved  any  of  three  properties:  (1)  problems  with  the  information  transfer  to  the  cockpit,  (2)  a  lack 
of  awareness  by  the  crew,  and/or  (3)  poor  spatial  judgment.  The  principle  causes  for  reported  runway 
transgressions  were  information  transfer  and  awareness.  It  was  often  difficult  to  differentiate  between 
these  in  practice.  However,  poor  spatial  judgment  was  not  found  to  be  a  significant  contributor  to 
runway  transgressions. 

KEYWORDS:  aviation  (civilian),  cognitive  processes,  measurement,  pilot  error, 
retrospective  measures,  subjective  measures 
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207.  Taylor,  R.M.  (1989,  October).  Situation  awareness:  Aircrew  constructs  for 
subjective  estimation.  (Institute  of  Aviation  Medicine  Report  No.  670). 
Farnborough,  UK:  RAF  Institute  of  Aviation  Medicine. 

This  paper  is  essentially  the  same  as  Taylor  (1990)  included  in  AGAjRD-CP-478  -  Situation  Awareness 
in  Aerospace  Operations  and  is  reviewed  under  that  citation. 
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208.  Taylor,  R.M.  (1990,  April).  Situational  awareness  rating  technique  (SART):  The 
development  of  a  tool  for  aircrew  systems  design.  In  AGARD-CP-478,  Situational 
Awareness  in  Aerospace  Operations  (pp.  3-1  to  3-17).  Neuilly  Sur  Seine,  France: 
Advisory  Group  for  Aerospace  Research  &  Development.  (AD-A223939) 

The  paper  includes  an  extensive  discussion  of  the  role  situation  awareness  plays  in  modem  aviation 
settings.  Definitional  issues  are  discussed.  In  the  conduct  of  the  research,  the  following  definition  of 
situational  awareness  is  used  (p.  3-3):  "Situational  awareness  is  the  knowledge,  cognition  and 
anticipation  of  events,  factors  and  variables  affecting  the  safe,  expedient  and  effective  conduct  of  the 
mission."  (The  authors  also  cite  (p.  3-2)  a  SA  definition  from  the  HQ  USAF  AFISC/SE  Safety 
Investigation  Workbook:  "Keeping  track  of  the  prioritized  significant  events  and  conditions  in  one's 
environment")  Knowledge  extraction  techniques  were  used  to  identify  concepts  that  RAF  aircrews  felt 
were  related  to  situation  awareness.  Analysis  showed  three  major  domains  were  related  to  situational 
awareness:  Attentional  Demands,  Attentional  Supply,  and  Understanding.  Ten  more  specific 
constructs  were  identified  as  underlying  the  three  domains.  These  constructs  and  domains  were  used 
to  define  the  dimensions  of  the  two  variants  of  the  Situational  Awareness  Rating  Technique  (i.e.,  the 
10-Dimensional  and  3-Dimensional  variants).  The  two  valiants  of  SART  allow  the  evaluator  to  select 
the  most  appropriate  tool  based  on  the  degree  of  intrusiveness  that  can  be  tolerated.  Research  testing 
the  SART  techniques  in  the  evaluation  of  real  tasks  is  suggested. 

KEYWORDS:  attention,  cognitive  processes,  definition,  measurement, 
questionnaire/survey  data,  SART,  subjective  measures 


209.  Taylor,  R.M.  (Ed.)  (1991,  December).  Situaticna!  awareness  in  dynamic  systems 
(Institute  of  Aviation  Medicine  Report  No.  708).  Farnborough,  UK;  RAF  Institute 
of  Aviation  Medicine. 

This  document,  consists  of  five  four-page  papers  presented  at  the  Special  Session  on  Situational 
Awareness  at  the  1 1th  Congress  of  the  International  Ergonomics  Association,  Paris,  France.  Although 
all  five  papers  were  written  in  an  aircrew  systems  context,  the  methods  and  findings  are  applicable  to 
systems  in  other  dynamic  environments. 

Situation  Awareness  in  Dynamic  Systems,  by  M.R.'Endsley,  focuses  on  situation  awareness  as  a 
critical  step  in  the  decision  making  process.  Designers  of  dynamic  systems  should  maximize  SA  and 
concentrate  on  "what  would  the  operator  ideally  like  to  know?" 

M.  Fracker  and  M.A.  Vidulich,  in  Measurement  of  Situation  Awareness:  A  Brief  Review,  explain  the 
three  major  approaches  to  assessing  SA.  The  authors  discuss  explicit,  implicit,  and  subjective  rating 
measures,  including  strengths,  weaknesses,  and  cautions  for  each  type  measure. 

In  Subjective  Measurement  of  Situational  Awareness,  R.M.  Taylor  and  S.  J.  Selcon  discuss  the  key 
problems  of  subjective  measures  of  SA:  validity  (how  well  a  technique  measures  what  it  is  actually 
intended  to  measure);  sensitivity  (capacity  of  a  measure  to  discriminate  variations  in  SA);  and 
diagnosticity  (how  well  a  measure  discriminates  causes  of  differences  in  SA).  The  authors  also  discuss 
their  Situational  Awareness  Rating  Technique  (SART),  including  its  strengths  and  weaknesses. 

In  Situation  Assessment  and  Situation  Awareness  in  a  Team  Setting,  K.L.  Mosier  and  T.R.  Chidester 
discuss  an  investigation  of  decision-making  strategies  employed  by  commercial  air  transport  crews. 

The  investigation  focused  on  indicators  of  situational  assessment,  represented  by  information 
solicitation  and  transfer  communications  occurring  during  simulated  inflight  emergencies. 

S.J.  Selcon  and  R.M.  Taylor  discuss  SA  in  the  context  of  a  decision  support  system  (DSS)  in 
Decision  Suoport  and  Situational  Awareness.  Their  paper  addresses  the  following  key  issues  of  the 
expert  system/  human  operator  relationship;  trust  in  the  system;  the  representation  of  uncertainty; 
communication;  and  appropriate  level  of  autonomy. 

KEYWORDS;  aviation  (civilian),  cockpit  resource  management  (CRM),  cognitive 
processes,  decision  aiding,  decision  making,  implicit  measures, 
measurement,  memory  probe  measures,  pilot  aiding,  SAGAT,  SART, 
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210.  Taylor,  R.M.,  and  Saloon,  S.J.  (1990a).  Cognitive  quality  and  situational 

awareness  with  advanced  aircraft  attitude  displays.  In  Proceedings  of  the  Human 
Factors  Society  34th  Annuai  Meeting  (Volume  1,  pp.  26-30).  Santa  Monica,  CA: 
Human  Factors  Society. 

This  paper  presents  two  studies  comparing  "cognitive  quality"  and  measures  of  SA  with  four  different 
types  of  attitude  indicator  displays.  In  both  studies,  attitude  was  presented  tachistoscopically  by 
depictions  of  HUD,  an  ADI  ball,  an  "outside-in"  picture  of  an  F-18  model,  and  a  pictorial  command 
indicator,  viewed  line  astern  as  by  a  "buddy"  aircraft,  again  using  F-18  model  pictures.  In  the  first 
study,  subjects  (16  student  pilots)  were  asked  to  make  a  stick  input  to  recover  to  straight  and  level 
flight.  Time  and  direction  of  initial  stick  inputs  were  measured,  along  with  subjective  SA  via  the 
Situational  Awareness  Rating  Technique  (SART).  The  same  measures  were  used  for  Study  2,  but 
subjects  were  shown  two  pictures  and  asked  to  return  the  aircraft  to  the  attitude  shown  in  the  first  one. 

KEYWORDS:  attitude  displays,  cognitive  processes,  decision  making,  display  design, 

measurement,  performance-based  measures,  pilot  aiding,  SART,  subjective 
measures 
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211.  Taylor,  R.M.,  and  Selcon,  S.J.  (1990b),  Understanding  situational  awareness.  In 
E.J.  Lovesey  (Ed.),  Proceedings  of  the  Ergonomics  Society's  1990  Annual 
Conference  (pp.  105-1 1 1 ).  London:  Taylor  and  Francis. 

This  is  a  theoretical  paper  which  presents  concepts  and  summarizes  work  performed  at  the  RAF' 
Institute  of  Aviation  Medicine  (I  AM)  in  the  field  of  SA.  Taylor  and  Selcon  posit  that  SA  is  a  global 
paradigm  which  gives  us  a  framework  for  studying  problems,  issues,  and  solutions  associated  with 
aircrew  systems  design.  Studies  are  summarized  which  demonstrate  the  benefit  of  "redundancy  gain" 
and  "cognitive  integrality"  in  HUD  pitch  ladder  design  features.  Intelligent  decision-aiding  for  the 
pilot,  for  helping  to  resolve  uncertainty  and  therefore  reduce  workload,  is  discussed  as  a  technology 
which  is  ripe  for  SA  applications.  A  recent  study  at  the  lAM  showed  that  decision-making  time  was 
reduced  when  "probability  tags"  were  presented  for  different  options.  The  process  undertaken  to 
develop  the  Situational  Awar .ness  Rating  Technique  (SART)  is  outlined  briefly;  because  this 
measurement  technique  is  derived  from  aircrew  knowledge  and  experience,  the  authors  feel  that  the 
constructs  wliich  make  up  the  SART  are  in  effect  a  definition  of  S A.  Taylor  and  Selcon  mention  three 
studies  which  "showed  that  SART  had  utility  in  predicting  performance  on  skill,  rule,  and  knowledge 
based  tasks;"  these  studies  are  discussed  in  greater  detail  in  Selcon  and  Taylor,  1989.  The  authors 
conclude  by  proposing  that  SART  is  a  noteworthy  concept  because  it  provides  definition  along  with  a 
measurement  tool  which  can  be  used  effectively  in  the  system's  design  process.  They  do  stipulate  that 
more  research  should  be  done  to  "develop  a  method  of  conjoint  scaling  for  the  SART  dimensions." 

KEYWORDS:  attitude  displays,  cognitive  processes,  decision  aiding,  decision  making, 
definition,  display  design,  measurement,  pilot  aiding,  SART,  subjective 
measures,  workload 
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212.  Taylor,  R.M.,  and  Selcon,  S.  (1991).  Subjective  measurement  of  situational 

awareness.  In  Designing  for  Everyone:  Proceedings  of  the  11th  Congress  of  the 
International  Ergonomics  Association  (Volume  1,  pp.  789-791).  Paris,  France: 
Taylor  &  Francis. 

This  paper,  along  with  4  other  SA  related  papers  presented  at  the  Congress,  were  reprinted  in  Taylor 
(1991)  and  is  reviewed  under  that  listing. 

KEYWORDS:  measurement,  SART,  subjective  measures 


213.  Taylor,  R.M.,  and  Selcon,  S.J.  (1994).  Situation  In  mind:  Theory,  application  and 
measurement  of  situational  awareness.  In  R.D.  Gilson,  D.J.  Garland,  and  J.M. 
Koonce  (Eds.),  Situational  awareness  in  complex  systems:  Proceedings  of  a 
CAHFA  Conference  (pp.  69-77).  Daytona  Beach,  FL:  Embry-Riddle  Aeronautical 
University  Press. 

The  authors  use  a  "question  and  answer"  format  to  address  what  they  perceive  to  be  the  key  issues  in 
SA.  The  four  main  questions  discussed  are  (p.  69): 

"(1)  What  is  the  problem? 

(2)  What  is  the  relevant  theory? 

(3)  What  are  the  consequences  of  the  theory? 

(4)  Do  the  theoretical  predictions  fit  experience  and  the  experimental  data?" 

The  problem  is  said  to  be  that  lack  of  S A  is  often  cited  as  a  causal  factor  in  errors  and  reduced  mission 
effectiveness.  The  influence  of  SA  (or  its  lack)  on  system  performance  show  up  in  numerous  different 
ways  that  seem  to  make  defining  SA  very  difficult.  The  authors  review  examples  of  settings  in  which 
the  SA  concept  has  been  evoked  and  various  approaches  to  defining  SA.  The  authors  own  approach  to 
defining  SA  depended  on  an  empirical  process  of  extracting  notions  of  S.A  from  pilots  (see  Taylor, 
1989).  In  discussing  theories  of  SA,  the  authors  point  out  that  most  theories  of  SA  have  utilized  the 
following  three  basic  psychological  concepts:  limited  attentional  resources,  working  memory  (or  short 
term  memory),  and  long  term  memory  (especially  schema  in  long  term  memory).  However,  the  authors 
suggest  that  several  other  psychological  theories  could  be  applied  to  the  problem  of  SA.  These 
additional  theories  include  attentional  priority,  perceptual  organization,  perceptual  control  theory, 
mental  models,  semantic  memory,  spatial  information  processing,  dynamic  mental  representation, 
human  error,  naturalistic  decision  making,  and  situated  acts.  In  considering  the  consequences  of  the 
current  theoretical  state-of-affairs,  the  authors  state  (p.  73):  "Since  the  theory  of  SA  appears  selective 
and  fragmented,  the  consequences  are  that,  as  yet,  there  are  few  generalizable  conclusions  or 
predictions  that  seem  to  be  more  than  common  sense  at  best,  and  self-fulfilling  prophesies  at  worst." 

In  trying  to  answer  whether  the  SA  research  findings  fit  the  theoretical  predictions,  the  authors  suggest 
that  answers  to  this  question  are  limited  by  our  ability  to  measure  SA.  SART  is  presented  as  a 
potential  SA  metric. 

KEYWORDS:  attention,  cognitive  processes,  decision  making,  definition,  measurement, 
mental  models,  SART,  schema,  subjective  measures 
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214,  Tenney,  Y.J.,  Adanris,  M.J.,  Pew,  R.W.,  Huggins,  A.W.F.,  and  Rogers,  W.H. 

(1992,  July).  A  principled  approach  to  the  measurement  of  situation  awareness  in 
commercial  aviation  (NASA  Contractor  Report  4551).  Washington,  DC:  National 
Aeronautics  and  Space  Administration. 

This  report  covers  information  processing  theories  related  to  pilot  SA  as  well  as  several  suggested 
strategies  for  measuring  SA.  From  a  cognitive  standpoint,  Tenney  et  al.  distinguish  the  process  of  SA 
(situation  assessment)  from  the  product,  which  is  the  "state  of  awareness  with  respect  to  information 
and  knowledge."  Definitions  of  SA  are  also  categorized  according  to  this  dichotomy.  The  elaboration 
on  information  processing  theory  merges  Neisser's  theory  of  the  perceptual  cycle  with  principles  from 
Sanford  and  Garrod's  theory  of  memory  and  attention.  The  result  is  a  framework  which  nicely 
illustrates  pilot  assessment  and  awareness,  and  which  implies  that  the  product  continually  influences 
the  process.  Effect  of  expertise  in  overcoming  ambiguity  in  SA  is  treated  with  a  discussion  of  seven 
compensatory  strategies  used  by  highly  experienced  pilots  to  overcome  limits  of  performance  in 
multi-task  situations.  Then  specific  strategies  for  measuring  SA  are  presented  and  illustrated  through 
detailed  examples.  The  three  strategies  are;  (1)  measures  derived  from  scenario  manipulations,  (2) 
direct  measures  such  as  queries  and  recall  of  displays,  and  (3)  model-based  measures.  A  combination  of 
the  three  measures  is  suggested  as  an  effective  approach,  and  the  use  of  these  strategies  in  two  current 
studies  is  outlined.  Recent  aircraft  accident  reports  are  referenced  repeatedly,  relating  the  causes  of 
these  accidents  to  measurement  approaches.  Finally,  the  report  delves  into  effects  of  present-day 
cockpit  automation  on  pilot  S.\,  and  concludes  by  extrapolating  individual  pilot  SA  principles  into  the 
team  or  crew  domain.  This  report  will  be  of  use  to  researchers  interested  in  information-processing 
theories  as  applied  to  S  A,  and  to  those  who  seek  to  design  an  effective  suite  of  S  A  measures,  for  a 
variety  of  system  design  applications. 

KEYWORDS:  attention,  aviation  (civilian),  avionics,  cognitive  processes,  expertise, 
measurement,  memory  probe  measures,  performance-based  measures, 
schema,  team  SA 


215.  Ulbrich,  E.A.  (1990).  Situational  awareness  in  large  aircraft,  in  Proceedings  - 

Cockpit  Displays  and  Visual  Simulation  (SPIE  Proceedings  Series  Volume  1 289,  pp. 
2-10).  Bellingham,  WA:  Society  of  Photo-Optical  Instrumentation  Engineers. 

This  paper  discusses  the  possible  use  of  and  artificial  intelligence  system,  called  Adaptive  Network  for 
Avionics  Research  Management  (ANARM),  to  enhance  the  SA  of  pilots  flying  large  transport  aircraft 
in  a  high-threat  combat  environment.  In  discussing  the  development  of  the  program,  the  authors  state 
(p.  3):  "In  general,  as  the  number  of  threatening  aircraft  and  ground-based  threats  increased,  the 
system  evolved  from  a  pilot-advisory  system  to  a  closed-loop  operational  system  with  pilot  overview 
and  management.  To  enhance  a  pilot's  situational  awareness,  the  system  relies  heavily  on  interactive 
displays  and  controls."  Several  display  design  approaches  were  evaluated  for  this  system.  Among  the 
four  conclusions  that  are  listed  the  authors  cite  as  an  accomplishment  (p.  4),  "Increased  situational 
awareness  due  to  data  fusion  using  3-D  displays  and  user-variable  viewpoints.  This  is  probably  the 
most  important  of  the  four  accomplishments." 

KEYWORDS;  aviation  (military),  avionics,  data  fusion,  display  design,  pilot  aiding 
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216.  Venturino,  M.,  Hamilton,  W.L.,  and  Dvorchak,  S.R,  (1990,  April). 

Performance-based  measures  of  merit  for  tactical  situation  awareness.  In 
AGARD-CP-478,  Situational  Awareness  In  Aerospace  Operations  (pp.  4-1  to  4-5). 
Neuilly  Sur  Seine,  France:  Advisory  Group  for  Aerospace  Research  & 
Development.  (AD-A223939) 

Using  a  data  base  from  an  evaluation  of  a  new  missile  system  within  a  high-fidelity  simulation,  the 
authors  explored  the  inter-relationships  among  situation  awareness  ratings,  a  performance-based 
measure  of  merit,  and  an  overall  mission  effectiveness  score.  Situation  awareness  was  rated  by  both 
the  pilot  performing  the  mission  and  by  expert  observers  watching  the  mission.  Correlations  between 
self-reported  situation  awareness  and  mission  effectiveness  was  moderate.  Observer  ratings  appeared 
to  correlate  somewhat  more  strongly  with  mission  effectiveness.  Interestingly,  pilots  that  rated  their 
situation  awareness  as  low  or  average  tended  to  be  in  agreement  with  the  mission  effectiveness  score. 
However,  those  pilots  that  rated  their  situation  awareness  as  high  had  mission  effectiveness  scores 
ranging  from  poor  to  superior.  As  the  authors  point  out  (p.  4-3);  "Good  SA,  as  rated  by  the  pilot 
appears  to  be  a  necessary  but  not  sufficient  contributor  to  good  pilot  performance."  Predicting  overall 
mission  effectiveness  was  greatly  enhanced  when  the  situation  awareness  ratings  were  considered 
along  with  information  about  the  performance-based  measure  of  merit  (fire  point  selection). 

KEYWORDS:  aviation  (military),  combat  simulation,  measurement,  performance-based 
measures,  subjective  measures 


217.  Venturino,  M.,  and  Kunze,  R.J.  (1989).  Spatial  awareness  with  a  helmet-mounted 
display.  In  Proceedings  of  the  Human  Factors  Society  33rd  Annual  Meeting 
(Volume  2,  pp.  1388-1391).  Santa  Monica,  CA:  Human  Factors  Society. 

No  formal  definition  is  presented,  but  the  authors  propose  a  "crude  dichotomy"  for  decomposing 
situation  awareness;  state  awareness  of  aircraft  subsystems,  and  spatial  awareness  of  relevant  aircraft. 
The  paper  reported  two  laboratory  studies  of  human  spatial  awareness  within  a  virtual  world 
constructed  by  a  helmet-mounted  display  (HMD).  The  field-of-view  of  the  HMD  was  varied  along 
with  the  memory  load  of  potential  targets  that  had  to  be  relocated  back  to  their  original  position.  In 
general,  the  HMD  field-of-view  had  a  large  effect  on  the  amount  of  time  the  subject  required  to  study 
the  environment,  but  little  effect  on  the  accuracy  of  the  placement  of  the  target  objeas.  However, 
replacement  error  was  significantly  affected  by  increasing  the  memory  load  of  potential  targets.  The 
results  suggest  that  a  larger  field-of-view  penuits  faster  construction  of  good  spatial  awareness. 

KEYWORDS:  definition,  display  design,  helmet-mounted  displays  (HMDs),  measurement, 
performance-based  measures 
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218.  Vidulich,  M.A.,  Crabtree,  M.S.,  and  McCoy,  A.L.  (1993).  Developing  subjective 
and  objective  metrics  of  pilot  situation  awareness.  In  Proceedings  of  the  Seventh 
International  Symposium  on  Aviation  Psychology  (Volume  2,  pp.  896-900). 
Columbus:  The  Ohio  State  University. 

This  study  compares  techniques  used  in  assessing  situational  awareness.  The  authors  based  their  work 
on  the  logic  of  early  mental  workload  studies.  Situation  awareness  was  measured  using  several 
different  tools.  After  a  minimum  of  16  hours  training,  twelve  college-aged  subjects  flew  three  sessions 
(two  or  three  blocks  per  session,  six  flights  per  block)  of  simulated  air-to-ground  attacks  in  the 
Simulator  for  Tactical  Operations  Research  and  Measurement  (STORM).  Each  flight  consisted  of  a 
low-level  route  terminating  in  a  cannon  attack  on  six  enemy  vehicles  ("tanks").  Display  conditions 
(tank  color,  target  display  in  aircraft)  were  varied  in  order  to  affect  the  difficulty  in  obtaining  SA. 
Objective  S  A  was  measured  by  explicit  memory  probes  (freezing  simulation  and  asking  for  relevant 
information)  and  implicit  probes  (based  on  signal  detection  task  measures  of  effective  cannon  use). 
Subjective  SA  measures  consisted  of  the  10-dimensional  Situation  Awareness  Rating  Technique 
(Si^T),  the  Overall-SART  Scale,  a  unidimensional  Overall-SA  scale,  and  the  SA  Subjective  Workload 
Dominance  (SA-SWORD)  technique.  The  results  of  the  implicit  probe  are  not  reported  here,  while  the 
results  of  the  memory  probe  were  labeled  as  "disappointing."  Overall-SA  measures  did  not  show  any 
display  effects;  the  authors  state  that  this  "implies  that  a  simple  unidimensional  SA  scale  is  not  an 
effective  SA  measure."  The  Overall-SART  ratings  were  a  sensitive  metric  but  lacked  a  statistically 
significant  test-retest  reliability. 

KEYWORDS:  air-to-ground  mission,  aviation  (military),  combat  simulation,  implicit 

measures,  measurement,  memory  probe  measures,  performance-based 
measures,  reliability,  SART,  SA-SWORD,  subjective  measures 


219.  Vidulich,  M.A.,  and  Hughes,  E.R.  (1991).  Testing  a  subjective  metric  of  situation 
awareness.  In  Proceedings  of  the  Human  Factors  Society  35th  Annual  Meeting 
(Volume  2,  pp.  1307-1311).  Santa  Monica,  CA:  Human  Factors  Society. 

The  purpose  of  the  research  described  in  this  paper  was  to  test  a  subjective  situation  awareness  (SA) 
metric,  the  SA  Subjective  WORkload  Dominance  (SWORD)  technique.  This  technique  applied 
SWORD  workload  assessment  principles  to  an  evaluation  of  two  different  cockpit  displays.  The 
subjects,  twelve  Air  Force  pilots,  used  Fire  Control  Radar  (FCR)  and  Horizontal  Situation  Format 
(HSF)  displays  during  eight  simulator  flights,  four  flights  per  display.  The  flights  consisted  of  ingress 
(low  threat:  no  surface-to-air  missile  (SAM)  attacks,  enemy  armed  with  short-range  nussiles)  and 
engagement  (beyond  visual  range,  multiple-threat  combat,  high  threat:  SAM  attacks,  enemy  fighters 
with  advanced  long-range  missiles).  After  the  trials,  the  subjects  performed  SA-SWORD  ratings  on 
combinations  of  display,  segment  and  threat.  Pilots  rated  the  HSF  as  providing  better  SA.  The  authors 
state  that  this  study  was  a  first  attempt  to  use  SA-SWORD  as  an  SA  metric  and  that  they  have  not 
reached  a  final  conclusion  concerning  test  sensitivity.  The  test  did  show,  however,  good  SA-SWORD 
sensitivity  to  display  manipulation  and  good  inter-rater  reliability,  suggesting  that  SA-SWORD  can  be 
useful  as  a  subjective  tool.  Pilot  comments  regarding  the  value  of  SA-SWORD  were  generally  neutral 
or  positive. 

KEYWORDS:  air-to-air  mission,  aviation  (military),  combat  simulation,  measurement, 
reliability,  SA-SWORD,  subjective  measures,  workload 
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220.  VIdulich,  M.A.,  Stratton,  M.,  Crabtree,  M.,  and  Wilson,  G.  (1994). 

Performance-based  and  physiological  measures  of  situational  awareness. 
Aviation,  Space,  jrnd  Environmental  Medicine,  65(5,  Supplement),  A7-A12. 

This  paper  is  a  more  detailed  presentation  of  portions  of  the  project  discussed  in  Stratton,  Wilson,  and 
Crabtree  (1993)  and  Vidulich,  Crabtree,  and  McCoy  (1993). 

KEYWORDS:  air-to-ground  mission,  aviation  (military),  combat  simulation,  implicit 

measures,  measurement,  memory  probe  measures,  performance-based 
measures,  physiological  measures,  SART,  subjective  measures,  workload 


221.  Waag,  W.L.,  and  Houck,  M.R.  (1994).  Tools  for  assessing  situational  awareness  In 
an  operational  fighter  environment.  Aviation,  Space,  and  Environmental  Medicine, 
55(5,  Supplement),  A13-A19. 

This  article  is  a  detailed  review  of  one  of  the  essential  components  of  the  Armstrong  Laboratory's 
Situation  Awareness  Integration  (SAINT)  team's  research  project.  The  authors  recount  the  extensive 
development  and  data  collection  work  that  was  performed  to  create  a  S  A  criterion  measure  for 
assessing  over  200  F-IS  pilots.  This  criterion  measure  is  called  the  Situational  Awareness  Rating 
Scales  (SARS).  The  SARS  were  developed  in  several  parallel  forms;  self-rating,  peer  rating,  and 
supervisor  rating.  Reliability  of  the  SARS  was  found  to  be  strongly  positive.  Also,  the  SARS  was 
found  to  be  significantly  related  to  flight  experience  and  current  flight  qualification.  A  subset  of  the 
SARS  evaluated  F-IS  pilots  were  selected  for  a  series  of  simulated  combat  engagements.  Tliis  planned 
follow-on  should  provide  an  opportunity  to  assess  the  predictive  validity  of  the  SARS  scores,  please 
refer  to  Grant  McMillan's  paper  in  this  report  for  an  overall  review  of  the  SAINT  team  research 
program.) 

KEYWORDS:  aviation  (military),  measurement,  subjective  measures 


222.  Waddell,  D.  (1979,  Winter).  Situational  awareness.  USAF Fighter  Weapons 
Review,  pp.  3-6, 

After  noting  that  SA  had  become  a  commonly  discussed  issue  during  debriefings,  the  author  evaluates 
the  concept  of  SA.  A  simple  definition  is  offered  (p.  3),  "Situational  awareness  is  an  assessment  of  a 
situation  based  on  the  best  possible  information."  The  author  breaks  SA  doAvn  into  three  categories: 
Actual  (based  on  direct  sensory  information).  Extrapolated  (mentally  extrapolating  changes  in  aspects 
of  the  situation  that  are  not  at  the  moment  available  to  direct  sensory  input),  and  Best  Guess  (using 
predictability  to  generate  rules  of  thumb).  The  author  proposes  methods  for  improving  pilot  SA.  Most 
suggestions  involve  tactics,  or  training  ideas.  The  use  of  debriefing  to  increase  S  A  for  fUture  flighters 
and  to  identify  possible  training  needs  is  highlighted. 

KEYWORDS:  aviation  (military),  cognitive  processes,  definition,  measurement,  mental 
models,  retrospective  measures,  subjective  measures,  training 
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223.  Ward,  G.F.,  and  Hassoun,  J.A.  (1990,  July).  The  effects  of  head-up  display 
(l-IUDI  pitch  ladder  articulation,  pitch  number  location  and  horizon  line  length  on 
unusual  attitude  recoveries  for  the  F-16  (Tech.  Report  No.  ASD-TR-90-5008). 
Wright-Patterson  Air  Force  Base,  OH;  Aeronautical  Systems  Division. 
(AD-A268666) 

Along  with  assessing  the  impact  of  different  HIJD  formats  on  the  attitude  recovery  performance  and 
subjective  ratings  of  workload,  the  authors  included  a  subjective  situation  awareness  scale.  The 
subjective  awareness  scale  was  a  nine-point  scale  with  verbal  descriptors  ranging  from  "Decidedly 
DISAGREE"  to  "Decidedly  AGREE."  The  pilots'  responses  were  converted  to  numerical  values  based 
on  the  recommendations  from  Ai  my  Manual  P-71-1  (Training  and  human  factors  research  on  military 
systems:  Questionnaire  construction  manual).  After  randomly  selected  attitude  recoveries,  the  pilot 
was  asked  to  use  the  scale  to  respond  to  the  statement  (p.  10):  "I  experienced  no  confusion  with  this 
pitch  ladder  configuration  and  was  easily  able  to  recover  to  straight  and  level  flight."  The  scale  was 
sensitive  to  the  different  HUD  pitch  ladder  configurations,  but  the  format  that  was  rated  the  best  for 
attitude  awareness  was  also  the  format  that  produced  the  greatest  number  of  inverted  recoveries.  As 
the  authors  put  it  (p.  30):  "Based  on  the  SA  results,  the  pilots  felt  that  their  situation  awareness  was 
very  good  when  flying  with  the  fully  articulated  pitch  scale  while  their  performance  unequivocally 
indicated  the  opposite....  These  results  indicate  that,  in  its  present  form,  the  SA  tool  is  not  a  sufficient 
indicator  of  pilot  situation  awareness." 

KEYWORDS:  attitude  displays,  aviation  (military),  display  design,  measurement, 

performance-based  measures,  pilot  aiding,  questionnaire/survey  data, 
retrospective  measures,  subjective  measures,  workload 


224.  Weinstein,  L.F.,  Ercoline,  W.R.,  Evans,  R.H.,  and  Bitton,  D.F.  (1992).  Head-up 
display  standardization  and  the  utility  of  analog  vertical  velocity  information  during 
instrument  flight.  The  International  Journal  of  Aviation  Psychology,  2,  245-260. 

As  part  of  a  program  to  create  a  standard  symbol  set  for  U.S.  Air  Force  Head-Up  Displays  (HUDs), 
this  project  examined  different  ways  to  present  vertical  velocity  information  on  the  HUD.  Performance 
and  subjective  data  both  suggested  that  combining  the  vertical  velocity  information  within  the  altimeter 
arc  was  best.  The  authors  suggest  that  a  beneficial  side-effect  of  this  would  be  a  reduction  of  display 
clutter.  This  was  seen  as  a  benefit  because  (p.  257),  "A  reduction  in  display  clutter  allows  the  pilot  to 
view  more  of  the  outside  world  through  the  HUD,  enabling  the  pilot  to  maintain  a  higher  level  of 
spatial  orientation  and  overall  situational  awareness." 

KEYWORDS:  aviation  (military),  display  design,  measurement,  performance-based 
measures,  pilot  aiding,  retrospective  measures,  subjective  measures 
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225.  Wells,  M.J.,  and  Venturino,  M.  (1989).  The  effects  of  increasing  task  complexity 
on  the  field-of-view  requirements  for  a  visually  coupled  system.  In  Proceedings  of 
the  Human  Factors  Society  33rd  Annual  Meeting  (Volume  1,  pp.  91-95).  Santa 
Monica,  CA:  Human  Factors  Society. 

Tlie  authors  tested  the  hypothesis  that  as  task  demands  were  increased  by  combining  a  visual  search 
and  replacement  task  with  a  tracking  task  in  a  dual-task,  the  value  of  a  larger  field-of-view  (FOV)  in  a 
head-coupled  helmet-mounted  display  would  increase.  ^  One  measure  of  the  FOV  requirement  was 
"spatial  awareness,"  which  was  assessed  by  target  replacement  accuracy.  The  authors  concluded  (p. 
95),  "Despite  the  increased  demands  imposed  by  the  small  FOVs,  subjects  managed  to  develop  the 
same  degree  of  spatial  awareness,  as  evidenced  by  the  replacement  error,  as  with  the  large  FOVs.  But 
at  a  cost.  The  cost  was  an  increase  in  tracking  error  with  the  small  FOVs." 

KEYWORDS:  display  design,  helmet-mounted  displays  (HMDs),  measurement, 
performance-based  measures 


226.  Wells,  M.J.,  Venturino,  M,,  and  Osgood,  R.K.  (1988).  Using  target  replacement 
performance  to  measure  spatial  awareness  in  a  helmet-mounted  simulator.  In 
Proceedings  of  the  Human  Factors  Society  32nd  Annual  Meeting  (Volume  2,  pp. 
1429-1433).  Santa  Monica,  CA:  Human  Factors  Society. 

The  authors  studied  spatial  awareness  as  a  component  of  situational  awareness.  The  research  was 
conducted  within  a  virtual  world  environment  created  by  a  helmet-mounted  display  (HMD).  The 
experiment  evaluated  the  effects  of  field-of-view  size,  background,  number  of  target  objects,  and 
amount  of  search  time  on  the  performance  of  a  replacement  task  (i.e.,  replacement  of  objects  to  their 
original  location).  Results  showed  that  replacement  performance  was  sensitive  to  all  of  the 
independent  variables,  suggesting  that  it  was  a  good  metric  of  spatial  awareness. 

KEYWORDS:  display  design,  helmet-mounted  displays  (HMDs),  measurement, 
performance-based  measures 


227.  Whitaker,  L.A.,  and  Klein,  G.A.  (1988,  April).  Situation  awareness  in  the  virtual 
world:  Situation  assessment  report.  In  Proceedings  of  the  Psychology  in  the 
Department  of  Defense  Eleventh  Symposium  (Tech.  Report  No.  USAFA-TR-88-1, 
pp.  321-325).  Colorado  Springs,  CO:  US  Air  Force  Academy.  (AD-A1 98723) 

The  authors  propose  tools  for  the  evaluation  of  the  needs  of  pilots  working  with  future-generation 
virtual  world  cockpits.  The  tools  are  the  Critical  Decision  Method  ("a  pool  of  techniques  and  types  of 
probes  that  elicit  detailed,  specific  information  about  the  conscious  processes  and  strategies  that 
underlie  real-world  decision  making"  (p.323))  and  the  Situation  Assessment  Report  ("a  formalism  for 
dynamic  decisions  by  tracing  the  evolution  of  the  decision  maker's  situation  awareness  as  new  cues  and 
information  are  received"  (p.  324)).  The  tools  grew  out  of  a  research  program  studying  expert  decision 
makers  in  naturalistic  settings. 

KEYWORDS:  cognitive  processes,  decision  making,  expertise,  measurement,  retrospective 
measures,  subjective  measures 
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228.  Wickens,  C.D.  (1992,  December).  Workload  and  situation  awareness:  An  analogy 
of  history  and  implications,  INSIGHT:  The  Visual  Performance  Technical  Group 
Newsletter,  f4(4),  1-3. 

The  author  argues  that  the  concepts  of  workload  and  situation  awareness  are  similar  in  that  both 
concepts  evolved  out  of  the  operational  community  to  become  important  topics  in  the  research 
community  and  both  concepts  involve  an  inferred  mental  construct.  However,  whereas  workload  was 
the  "hot  issue"  of  the  1980s,  situation  awareness  appears  to  be  the  issue  of  the  1990s.  The  author 
suggests  a  definition  for  situation  awareness  (p.  2);  "Situation  awareness  refers  to  the  ability  to  rapidly 
bring  to  consciousness  those  characteristics  that  evolve  during  a  flight."  The  author  emphasizes  the 
word  "evolve"  in  order  to  distinguish  the  information  involved  in  situation  awareness  from  simple 
declarative  and  procedural  knowledge.  The  author  also  emphasizes  that  the  "ability  to  bring"  phrase 
refers  to  information  that  may  be  inunediately  available  and  important,  but  not  reside  in  consciousness 
at  the  moment.  It  is  suggested  that  the  overall  domain  of  situation  awareness  can  be  decomposed  into 
at  least  four  topics;  Navigation  awareness,  Systems  awareness.  Task  awareness,  and  Temporal 
awareness.  The  available  measurement  techniques  and  some  potential  application  areas  are  briefly 
discussed. 

KEYWORDS:  cognitive  processes,  definition,  measurement,  workload 


229.  Wickens,  C.D.,  and  Flach,  J.M.  (1988).  Information  processing.  In  E.L.  Wiener 
and  D.C.  Nagel  (Eds.),  Human  factors  in  aviation  (pp.  11 1-155).  San  Diego,  CA: 
Academic  Press. 

This  paper  provides  a  major  review  of  the  relevance  of  human  information  processing  theory  to 
aviation.  In  the  discussion  of  decision  making  in  aviation,  the  authors  suggest  a  model  that  includes 
"situation  assessment"  as  a  major  component  (see  Figure  S.2,  p.  128).  Situation  assessment  is  seen  to 
be  problematic  if  it  is  based  on  the  operation  of  heuristics  rather  than  based  on  an  appreciation  of  the 
reliability  of  the  various  cuts  present  in  a  situation.  Sununarizing  the  role  of  situation  assessment  in 
decision  making,  the  authors  state  (p.  132),  "Instead  of  using  cue  reliability  as  a  basis  for  choosing  their 
hypothesis,  people  more  often  focus  attention  most  heavily  on  those  cues  that  are  physically  salient..., 
and  those  that  are  likely  to  confirm  the  hypothesis  that  was  already  tentatively  formed....  If  those  cues, 
by  chance  or  by  design,  also  happen  to  be  quite  reliable,  then  the  assessment  of  the  situation  will 
likewise  be  accurate;  but  if  they  are  not  reliable,  and  their  indicated  diagnosis  is  wrong,  then  even  the 
best-intended  decision  of  what  action  to  take  may  lead  to  disaster  because  it  will  be  based  on  a  faulty 
assessment  of  the  world." 

KEYWORDS:  attention,  cognitive  processes,  decision  making 
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230.  Wiener,  E.L.  (1993).  Life  in  the  second  decade  of  the  giass  cockpit.  In 

Proceedings  of  the  Seventh  International  Symposium  on  Aviation  Psychology, 
(Volume  1,  pp.  1-7).  Columbus:  The  Ohio  State  University. 

In  t)ie  course  of  a  keynote  speech  on  the  histoiy  and  design  of  glass  cockpits,  the  author  touches  briefly 
upon  the  topic  of  SA.  The  author  takes  issue  with  the  common  tendency  of  researchers  to  propose  that 
advanced  avionics  in  the  commercial  airline  cockpit  might  degrade  S  A.  One  problem  noted  by  the 
author  is  the  lack  of  a  clear  definition  of  SA.  The  author  suggests  that  (p.  4),  "you  can  bring  a  panel 
discussion  on  the  subject  to  a  stand-still  by  merely  asking  the  speaker  or  the  panel  what  they  mean  by 
the  'situational  awareness'."  The  author  contends  that  SA  is  usually  enhanced  by  the  automation  in 
glass  cockpits.  For  example  (p.  4),  "Think  of  the  ability  to  instantly  locate  the  closest  adequate  airfield 
in  the  event  of  an  extreme  emergency.  I  have  a  name  for  what  can  be  obtained  from  that  type  of 
information;  I  call  it  situational  awareness."  Other  similar  examples  are  cited. 

KEYWORDS:  aviation  (civilian),  avionics,  definition,  pilot  aiding 


231.  Wise,  J.A.,  Guide,  P.C.,  Abbott,  D.W.,  and  Ryan,  L.  (1993).  Automated  corporate 
cockpits;  Some  observations.  In  Proceedings  of  the  Human  Factors  and 
Ergonomics  Society  37th  Annual  Meeting  (Volume  1,  pp.  6-10).  Santa  Monica, 

CA:  Human  Factors  and  Ergonomics  Society. 

This  article  discusses  the  results  of  a  survey  to  determine  the  effects  of  automation  in  the  cockpits  of 
corporate  aircraft.  The  surveys  were  distributed  to  about  one-quarter  of  the  population  of  corporate 
pilots.  A  change  in  SA  as  a  function  of  automation  was  mentioned  by  some  of  the  respondents.  The 
authors  stated  the  S  A  result  as  (p.  8),  "many  comments  report  that  automation  has  contributed  to 
better  situational  awareness  (which  they  seem  to  only  think  of  in  terms  of  navigation);  increasing  their 
ability  to  become  more  aware  of  the  aircraft's  status  in  real  time," 

KEYWORDS:  aviation  (civilian),  avionics,  measurement,  pilot  aiding,  questionnaire/survey 
data,  subjective  measures 


232.  Zacharias,  G.L.,  Miao,  A.X.,  Riley,  E.W.,  and  Osgood,  R.K.  (1992,  November). 
Situation  awareness  metric  for  cockpit  configuration  evaluation  (Tech.  Report  No. 
AL-TR- 1993-0042).  Wright-Patterson  Air  Force  Base,  OH:  Armstrong  Laboratory. 
(AD-B1 76582) 

This  report  reviews  the  development  of  a  computational  model  approach  for  evaluating  the  impact  of 
cockpit  display  changes  on  pilot  SA.  Enabling  technologies  that  were  evaluated  for  possible 
contribution  to  the  model  included:  modem  estimation  models,  artificial  neural  networks,  Bayesian 
belief  networks,  expert  systems,  and  object-oriented  system  description  languages.  A  proof-of-concept 
demonstration  of  the  approach  was  used  to  evaluate  the  impact  of  a  decision  aid  on  performance  of  a 
beyond  visual  range  (BVR)  air-to-air  engagement.  Recommendations  for  follow-on  efforts  are 
discussed. 

KEYWORDS:  air-to-air  mission,  aviation  (military),  decision  aiding,  decision  making, 
measurement,  modeling,  pilot  aiding 
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233.  Zenyuh,  J.P.,  Reising,  J.M.,  Walchli,  S.,  and  Biers,  D.  (1988).  A  comparison  of  a 
stereographic  3-D  display  versus  a  2-0  display  using  an  advanced  air-to-air  format. 
In  Proceedings  of  the  Human  Factors  Society  32nd  Annual  Meeting  (Volume  1,  pp. 
53-57).  Santa  Monica,  CA:  Human  Factors  Society. 

This  research  examined  whether  the  use  of  stereoscopic  displays  would  enhance  the  development  of 
pilot  SA.  SA  was  defined  as  (p.  S3),  "the  ability  to  visually  search  a  spatial  quadrant  of  the  air-to-air 
situation  display  relative  to  the  ownship  symbol  and  identify  the  aircraft  symbols  in  a  given  target 
group  found  in  that  quadrant."  The  research  was  conducted  in  laboratory  simulator.  The  subjects 
performed  the  visual  search  task  while  flying  the  aircraft  under  simulated  Instrument  Flight  Rules  (IFR) 
conditions.  Performance  measures  from  both  tasks  improved  when  the  presentation  of  the  visual 
search  task  was  stereoscopic.  This  suggests  that  the  inclusion  of  stereoscopic  information  facilitated 
the  development  of  S A  from  the  display. 

KEYWORDS:  air-to-air  mission,  aviation  (military),  combat  simulation,  definition,  display 
design,  measurement,  performance-based  measures,  pilot  aiding,  target 
identification 
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KEYWORD  GLOSSARY 


3-D  Audio  Displays  -  The  aiticle  discusses  the  potential  use  of  3-D  audio  displays  to  improve  SA. 


Air  Combat  Maneuvering  (ACM)  -  The  article  discusses  the  role  of  SA  in  ACM. 

Air  Traffic  Control  -  The  article  discusses  SA  in  air  traSic  control. 

Air-to-air  Mission  -  The  article  discusses  the  role  of  SA  in  air-to-air  combat  and/or  uses  a  moderate  to  high 
fidelity  simulation  of  air-to-air  combat  as  an  experimental  task. 

Air-to-ground  Mission  -  The  article  discusses  the  loie  of  SA  in  air-to-ground  combat  or  uses  a  moderate  to 
high  fidelity  simulation  of  air-to-ground  combat  as  an  experimental  task. 

Attention  -  The  paper  discusses  the  relationship  between  attention  and  SA 

Attitude  Displays  -  The  article  discusses  the  effects  of  aircraft  attitude  displays  on  pilot  SA. 

Automation  -  Refer  to  Avionics  for  applications  of  automation  to  aircraft.  Also,  many  of  the  Air  Traffic  Control 
citations  discuss  an  application  of  automation. 

Aviation  (Civilian)  •  The  article  discusses  SA  in  the  cockpit  of  non-militaiy  fixed-wing  commercial  aircraft 
This  would  include  general  aviation,  commercial  aviation,  corporate  aviation  or  any  other  article  that  rtigfintsrd  SA 
in  aviation  that  did  not  specify  a  military  context. 

Aviation  (Military)  -  The  article  discusses  SA  in  the  cockpit  of  fixed-wing  military  aircraft. 

Avionics  -  The  article  discusses  the  effects  of  aircraft  avionics  on  pilot  SA  or  discusses  the  application  of 
avionics  technology  to  non-aircraft  systems. 


Battlefield  SA  -  The  article  discusses  the  role  of  SA  in  land-based  Army  battles. 

Big  Picture  Displays  -  The  article  discusses  the  possible  effects  of  large-area  displays  on  operator/pilot  SA. 

Cockpit  Resource  Management  (CRM)  -  The  article  discusses  the  effects  of  CRM  on  aircrew  SA. 

Cognitive  Processes  -  The  article  proposes  or  discusses  theories  or  some  hypothesis  concerning  the  role  of 
cognitive  process  in  creating  or  maintaining  SA. 

Combat  Simulation  -  The  article  discusses  an  SA  experiment  that  was  perfonned  using  a  combat  simulator  of 
moderate  to  high  fidelity. 

Crew  Resource  Management  (CRM)  -  Refer  to  Coclq>U  Resource  Management  (CRM) 
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Data  Fusion  -  The  article  discusses  the  effects  of  data  fusion  on  SA.  Data  fusion  is  the  combination  of 
information  from  multiple  sources  into  one  display. 

Data  link  •  The  article  discusses  the  effects  of  data  link  on  SA.  Data  Links  are  digital  communications  from 
outside  an  aircraft  that  are  used  by  the  aircraft's  avionics  and/or  presented  on  the  aircraft's  displays.  Potential 
sources  for  the  conununications  include  other  aircraft,  air  traffic  control,  ground  stations,  or  satellites. 

Decision  Aiding  -  The  article  discusses  the  use  of  technology  to  improve  SA  and  thus  make  mote  effective 
decisions  and/or  make  decisions  more  easily. 

Decision  Making  -  The  article  discusses  the  relationship  between  SA  and  the  cognitive  process  of  decision 
making. 

Definition  -  The  article  either  proposes  a  specific  definition  for  SA  or  substantively  discusses  components  of 
SA. 

Design  Guidelines  •  The  article  proposes  specific  guidelines  for  improving  SA  in  human-machine  systems. 

Display  Design  -  The  article  discusses  some  aspect(s)  of  display  design  that  affects  SA  in  human-machine 
systems. 


Expertise  -  The  article  discusses  the  relationship  between  expertise  and  SA. 
Explicit  Measures  -  'Ratmo  Memory  Probe  Measures. 


Flight  Test  -  The  article  discusses  the  role  of  SA  in  the  design  or  measurement  of  a  flight  test  program  or 
discusses  observations  about  SA  that  resulted  from  a  test  flight  of  an  aircraff . 

Fratricide  -  The  article  discusses  the  possible  link  between  SA  deficiencies  and  combat  fiatricide  (i.e.,  attacks 
upon  fiiendly  forces). 


Helicopter  •  The  article  discusses  the  role  of  SA  in  the  operation  of  rotary-wing  aviation. 

Helmet-Mounted  Displays  (HMDs)  -  The  article  discusses  the  potential  effects  of  helmet-mounted 
displays  (HMDs). 


Implicit  Measures  -  The  article  either  substantively  discusses  the  potential  usefulness  of  implicit  signal 
detection  measures  of  SA  or  reviews  an  evaluation  that  used  an  implicit  measure  of  SA.  The  signal  may  be  either 
an  additional  arbitrary  sigrml  or  an  embedded  aspect  of  the  task. 

Individual  Differences  -  The  article  cither  presents  results  of  an  evaluation  of  using  selection  tests  to  identify 
individuals  with  generic  abilities  conducive  to  good  SA  or  substantively  discusses  the  prospects  for  such  tests. 
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Measurement  -  The  article  either  presents  results  from  a  SA  evaluation  using  either  specific  SA  measures  or  a 
general  approach  to  SA  measurement  that  could  be  applied  in  other  settings,  or  the  article  substantively  discusses 
important  issues  for  the  measurement  of  S  A. 

Memory  Probe  Measures  •  The  article  either  substantively  discusses  the  potential  uselulness  of  explicit 
memory  probe  measures  of  SA  or  reviews  an  evaluation  that  used  memory  probes  as  a  measure  of  SA. 

Mental  Models  -  The  article  discusses  the  relationship  between  mental  models  (or  some  similar  form  of 
internal  mental  representation)  and  SA. 

Modeling  -  The  article  discusses  a  model  that  attempts  to  predict  the  SA  achievable  based  on  a  set  of  attributes 
of  the  sibration  or  the  system  operator. 

Multiple  Resource  Theory  •  The  article  discusses  the  relationship  between  multiple  resource  theory  and  SA. 


Naval  SA  -  The  article  disaisses  the  role  of  S  A  in  water-based  Naval  battles. 


Performance-Based  Measures  •  The  article  either  substantively  discusses  the  potential  usefulness  of 
objective  task  performance  as  a  measure  of  SA  or  reviews  an  evaluation  that  used  objective  task  performance  as  a 
measure  of  SA.  In  some  cases,  the  performance  measures  might  be  used  as  a  criterion  forjudging  the  validity  of 
other  SA  measures. 

Physiological  Measures  -  Ttie  article  either  substantively  discusses  the  potential  use.fulness  of  physiological 
measures  of  SA  or  reviews  an  evaluation  that  used  physiological  measures  to  measure  SA. 

Pilot  Aiding  -  The  article  discusses  how  some  aspect(s)  of  the  design  of  the  cockpit's  displays,  controls, 
avionics,  software  etc.  could  enhance  the  pilot's  SA.  In  some  cases,  the  article  might  discuss  how  to  avoid  the 
potential  SA  degradation  that  could  occur  from  cockpit  changes. 

Pilot  Error  •  The  article  discusses  the  relationship  between  pilot  SA  and  the  production  of  pilot  error. 


Questionnaire/Survey  Data  -  The  article  discusses  the  use  of  questionnaire  or  surveys  to  assess  SA  and/or 
reviews  SA  data  collected  by  questioiuiaire,  survey,  or  interviews. 


Reliability  -  The  article  presents  data  on  the  reliability  of  at  least  one  SA  measurement  technique. 

Retrospective  Measures  -  The  article  either  substantively  discusses  the  potential  uselulness  of  retrospective 
recall  of  a  task  or  mission  as  a  measure  of  SA  or  reviews  an  evaluation  that  used  retrospective  measures  as  a 
measure  of  SA.  These  are  similar  to  Questionnaire/Survey  Data,  but  differ  in  that  they  are  focused  on  specific 
events  (usually,  but  not  always,  events  within  the  experiment).  In  some  cases,  this  might  refer  to  reports  of  actual 
operational  incidents. 


Review  Paper  -  The  article  includes  substantive  comparative  review  of  the  results  and/or  concepts  related  to  S  A 
that  are  expressed  in  other  articles  by  other  authors. 


SAG  AT  -  The  article  either  substantively  discusses  the  potential  usefulness  of  the  Situation  Awareness  Global 
Assessment  Technique  (SAGAT). 

SART  -  The  article  either  substantively  discusses  the  potential  usefulness  of  the  Situation  Awareness  Rating 
Technique  (SART)  or  reviews  an  evaluation  that  used  SART  or  employs  the  technique  in  a  study. 

SA-SWORD  ■  The  article  either  substantively  discusses  the  potential  usefulness  of  the  Situation  Awareness 
variant  of  the  Subjective  WORkload  Dominance  technique  (SA-SWORD)  or  reviews  an  evaluation  that  used  SA> 
SWORD  as  a  measure  of  SA. 

Schema  -  The  article  discusses  the  relationship  between  schemata  and  SA. 

Signal  Detection  Probe  Measures  -  Refer  to  Impticti  Measures. 

Subjective  Measures  -  The  article  either  substantively  discusses  the  potential  usefulness  of  subjective 
measures  of  SA  or  reviews  an  evaluation  that  used  subjective  measures.  Subjective  measures  include 
questionnaires/survey  data,  retrospective  measures,  verbal  protocols,  or  subjective  ratings. 


Target  Identification  •  The  article  reviews  the  role  that  identifying  potential  targets  plays  in  maintaining  a 
pilot's  SA. 

Team  Decision  Making  •  The  article  discusses  the  contribution  played  by  SA  in  the  decision-making  of 
teams. 

Team  SA  •  The  article  discusses  the  factors  that  affect  the  shared  SA  across  multiple  members  of  a  team. 

Training  •  The  article  either  presents  results  of  an  evaluation  of  using  training  to  improve  SA  or  substaittively 
discusses  the  prospects  for  such  training.  The  training  program  might  either  be  aimed  at  improving  generic  SA 
abilities  of  the  individuals  or  might  be  aimed  at  improving  SA  within  a  specific  task  domain. 

Verbal  Protocols  -  The  article  cither  substantively  discusses  the  potential  usefulness  of  verbal  protocols  as  a 
measure  of  SA  or  reviews  an  evaluation  that  used  verbal  protocols  as  a  measure  of  SA. 


Workload  •  The  article  either  discusses  a  hypothesis  concerning  the  relationship  between  SA  and  mental 
workload  and/or  discusses  an  evaluation  that  used  both  SA  and  workload  measures  for  assessment. 
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